00010000:00010000:1.0:1789760686.107870:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.107987:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.107997:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760686.108002:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.108010:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.108029:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.108038:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15716352->15720447] (req 15716352->15720447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760686.108132:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507114880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 118667us (118905us total) trans 4294971157 rc 0/0 00000100:00100000:1.0:1789760686.108142:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3867 00000100:00100000:1.0:1789760686.108152:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 667234656583 time 667304752337 state 0x2 ref 0 00000100:00100000:1.0:1789760686.115350:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760686.227612:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971158 is committed 00000001:00080000:1.0:1789760686.343952:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971159 is committed 00000100:00100000:1.0:1789760686.360248:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507115264 00000100:00100000:1.0:1789760686.360260:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507115264 00000100:00100000:1.0:1789760686.360315:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 667431178326 time 667558589568 state 0x0 ref 1 00000100:00100000:1.0:1789760686.360332:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 667558849984 time 667558589568 state 0x1 ref 1 00000100:00100000:1.0:1789760686.360366:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 667558849984 time 667675669181 state 0x0 ref 1 00000100:00100000:1.0:1789760686.360379:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507115264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760686.360425:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15728640, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x7807 00010000:00010000:1.0:1789760686.360433:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b653b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.360450:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=667s v=(0 0) 00010000:00010000:1.0:1789760686.360455:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760686.360461:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b653b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15728640->15732735] (req 15728640->15732735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.360476:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b653b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15728640->15732735] (req 15728640->15732735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760686.482028:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760686.625144:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760686.658737:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760686.731313:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507116032 00000100:00100000:1.0:1789760686.731325:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507116032 00000100:00100000:1.0:1789760686.731381:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 667974487972 time 668012338783 state 0x0 ref 1 00000100:00100000:1.0:1789760686.731397:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668012599199 time 668012338783 state 0x1 ref 1 00000100:00100000:1.0:1789760686.731463:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668012599199 time 668046766571 state 0x0 ref 1 00000100:00100000:1.0:1789760686.731476:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507116032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760686.731520:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15753216, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x8007 00010000:00010000:1.0:1789760686.731530:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6565 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.731545:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1789760686.731549:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760686.731555:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15753216->15757311] (req 15753216->15757311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.731570:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15753216->15757311] (req 15753216->15757311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760686.796337:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971167 is committed 00000100:00100000:1.0:1789760686.841149:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760686.880265:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760686.983381:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971172 is committed 00010000:00010000:1.0:1789760687.003954:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760687.019337:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971173 is committed 00002000:00010000:1.0:1789760687.020103:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15781888 -> 15785984 00002000:00010000:1.0:1789760687.020109:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760685 -> 1789760686 00002000:00010000:1.0:1789760687.020112:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760685 -> 1789760686 00010000:00010000:1.0:1789760687.020125:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020141:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020150:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020156:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760687.020160:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020169:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020793:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.020802:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6596 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15781888->15785983] (req 15781888->15785983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760687.020906:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507117568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31444us (31880us total) trans 4294971173 rc 0/0 00000100:00100000:1.0:1789760687.020916:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3883 00000100:00100000:1.0:1789760687.020925:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668268112418 time 668304749473 state 0x2 ref 0 00000001:00080000:1.0:1789760687.056651:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971174 is committed 00000100:00100000:1.0:1789760687.060585:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507117824 00000100:00100000:1.0:1789760687.060594:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507117824 00000100:00100000:1.0:1789760687.060657:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668305009889 time 668340679456 state 0x0 ref 1 00000100:00100000:1.0:1789760687.060671:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668340939872 time 668340679456 state 0x1 ref 1 00000100:00100000:1.0:1789760687.060725:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668340939872 time 668376028062 state 0x0 ref 1 00000100:00100000:1.0:1789760687.060738:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507117824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760687.060788:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15790080, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x8007 00010000:00010000:1.0:1789760687.060797:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b65a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.060814:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1789760687.060818:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760687.060824:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b65a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15790080->15794175] (req 15790080->15794175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.060838:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b65a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15790080->15794175] (req 15790080->15794175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760687.097004:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971175 is committed 00000100:00100000:1.0:1789760687.103300:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760687.151063:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971176 is committed 00000001:00080000:1.0:1789760687.233149:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971177 is committed 00000001:00080000:1.0:1789760687.362559:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971178 is committed 00000100:00100000:1.0:1789760687.369584:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760687.524039:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507118592 00000100:00100000:1.0:1789760687.524049:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507118592 00000100:00100000:1.0:1789760687.524106:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668686422163 time 668768722976 state 0x0 ref 1 00000100:00100000:1.0:1789760687.524121:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668768983392 time 668768722976 state 0x1 ref 1 00000100:00100000:1.0:1789760687.524149:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668768983392 time 668839453294 state 0x0 ref 1 00000100:00100000:1.0:1789760687.524161:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507118592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760687.524216:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15814656, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x8007 00010000:00010000:1.0:1789760687.524225:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b65ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.524251:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1789760687.524255:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760687.524261:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b65ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15814656->15818751] (req 15814656->15818751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.524274:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b65ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15814656->15818751] (req 15814656->15818751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760687.619857:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760687.697265:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668936004082 time 669012566058 state 0x0 ref 1 00000100:00100000:1.0:1789760687.697282:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507118976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760687.697334:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15826944, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x8007 00010000:00010000:1.0:1789760687.697344:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.697361:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:1.0:1789760687.697366:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760687.697372:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.697386:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.744379:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760687.744386:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.744395:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.744415:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.744423:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b65e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15826944->15831039] (req 15826944->15831039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.744787:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507118976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47505us (48333us total) trans 4294971184 rc 0/0 00000100:00100000:0.0:1789760687.744800:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3894 00000100:00100000:0.0:1789760687.744810:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668936004082 time 669012566058 state 0x2 ref 0 00000100:00100000:1.0:1789760687.749103:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760687.749207:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507119104 00000100:00100000:0.0:1789760687.749216:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507119104 00000100:00100000:0.0:1789760687.749271:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 668936004082 time 669012566058 state 0x0 ref 1 00000100:00100000:0.0:1789760687.749287:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669012826474 time 669012566058 state 0x1 ref 1 00000100:00100000:0.0:1789760687.749349:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669012826474 time 669064652145 state 0x0 ref 1 00000100:00100000:0.0:1789760687.749362:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507119104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760687.749408:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15831040, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x8007 00010000:00010000:0.0:1789760687.749417:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.749433:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760687.749438:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.749444:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.749459:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760687.787121:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971185 is committed 00002000:00010000:0.0:1789760687.787250:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15831040 -> 15835136 00010000:00010000:0.0:1789760687.787266:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787278:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787287:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787294:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760687.787299:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787309:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787334:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.787342:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15831040->15835135] (req 15831040->15835135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.787445:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507119104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38080us (38346us total) trans 4294971185 rc 0/0 00000100:00100000:0.0:1789760687.787456:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3895 00000100:00100000:0.0:1789760687.787466:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669012826474 time 669064652145 state 0x2 ref 0 00000100:00100000:1.0:1789760687.809448:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760687.849737:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971186 is committed 00000100:00100000:0.0:1789760687.856850:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507119360 00000100:00100000:0.0:1789760687.856861:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507119360 00000100:00100000:0.0:1789760687.856916:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669064912561 time 669125304815 state 0x0 ref 1 00000100:00100000:0.0:1789760687.856931:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669125565231 time 669125304815 state 0x1 ref 1 00000100:00100000:0.0:1789760687.856961:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669125565231 time 669172264512 state 0x0 ref 1 00000100:00100000:0.0:1789760687.856974:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507119360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760687.857024:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15839232, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x8007 00010000:00010000:0.0:1789760687.857033:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.857049:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760687.857054:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.857061:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15839232->15843327] (req 15839232->15843327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.857074:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b65f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15839232->15843327] (req 15839232->15843327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760687.907477:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971187 is committed 00000100:00100000:0.0:1789760687.916062:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760687.984330:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971188 is committed 00000100:00100000:0.0:1789760687.988868:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760688.032289:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971189 is committed 00002000:00010000:1.0:1789760688.077249:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15851520 -> 15855616 00010000:00010000:1.0:1789760688.077271:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077288:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077298:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077306:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760688.077312:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077329:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077358:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.077367:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b660d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15851520->15855615] (req 15851520->15855615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760688.077511:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507119744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40925us (41189us total) trans 4294971190 rc 0/0 00000100:00100000:1.0:1789760688.077526:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3900 00000100:00100000:1.0:1789760688.077549:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669304601773 time 669351870695 state 0x2 ref 0 00000100:00100000:0.0:1789760688.083097:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507119872 00000100:00100000:0.0:1789760688.083111:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507119872 00000100:00100000:0.0:1789760688.083203:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669304601773 time 669351870695 state 0x0 ref 1 00000100:00100000:0.0:1789760688.083224:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669352131111 time 669351870695 state 0x1 ref 1 00000100:00100000:0.0:1789760688.083270:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669352131111 time 669398572903 state 0x0 ref 1 00000100:00100000:0.0:1789760688.083285:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507119872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760688.083353:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15855616, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x8007 00010000:00010000:0.0:1789760688.083363:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6614 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.083561:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760688.083577:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.083587:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15855616->15859711] (req 15855616->15859711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.083605:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15855616->15859711] (req 15855616->15859711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760688.171320:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971192 is committed 00000100:00100000:1.0:1789760688.189315:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507120128 00000100:00100000:1.0:1789760688.189332:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507120128 00000100:00100000:0.0:1789760688.189501:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669398833319 time 669452203099 state 0x0 ref 1 00000100:00100000:0.0:1789760688.189611:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669452463515 time 669452203099 state 0x1 ref 1 00000100:00100000:0.0:1789760688.189683:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669452463515 time 669504984635 state 0x0 ref 1 00000100:00100000:0.0:1789760688.189721:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507120128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760688.189793:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15863808, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x8007 00010000:00010000:0.0:1789760688.189804:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.189824:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760688.189840:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.189847:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.189866:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760688.235682:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971193 is committed 00002000:00010000:0.0:1789760688.235814:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15863808 -> 15867904 00010000:00010000:0.0:1789760688.235836:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235853:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235863:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235871:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760688.235877:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235887:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235913:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.235921:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6622 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15863808->15867903] (req 15863808->15867903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.236042:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507120128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46317us (47542us total) trans 4294971193 rc 0/0 00000100:00100000:0.0:1789760688.236053:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3903 00000100:00100000:0.0:1789760688.236066:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669452463515 time 669504984635 state 0x2 ref 0 00000100:00100000:0.0:1789760688.244328:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760688.283564:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971194 is committed 00000100:00100000:1.0:1789760688.287147:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760688.342019:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971195 is committed 00000100:00100000:1.0:1789760688.346059:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760688.346280:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507120512 00000100:00100000:0.0:1789760688.346294:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507120512 00000100:00100000:0.0:1789760688.346443:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669560455672 time 669602678244 state 0x0 ref 1 00000100:00100000:0.0:1789760688.346465:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669602938660 time 669602678244 state 0x1 ref 1 00000100:00100000:0.0:1789760688.346549:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669602938660 time 669661850175 state 0x0 ref 1 00000100:00100000:0.0:1789760688.346565:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507120512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760688.346644:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15876096, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x8007 00010000:00010000:0.0:1789760688.346655:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6637 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.346676:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760688.346682:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.346689:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15876096->15880191] (req 15876096->15880191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.346904:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15876096->15880191] (req 15876096->15880191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.352385:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538249728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760688.352492:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760688.352595:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538249728 00000100:00100000:0.0:1789760688.352606:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538249728 00000100:00100000:0.0:1789760688.352675:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1789760688.352995:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538249856:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760688.353041:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760688.353084:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538249856 00000100:00100000:1.0:1789760688.353088:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538249856 00000100:00100000:0.0:1789760688.353155:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538249728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760688.353196:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3841982464 avail 3807379456 00000100:00100000:0.0:1789760688.353247:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538249728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (757us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760688.353257:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1789760688.358354:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:1.0:1789760688.358396:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538249856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760688.358438:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3841982464 avail 3807379456 00000100:00100000:1.0:1789760688.358549:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538249856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 180us (5507us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760688.358556:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:1.0:1789760688.358798:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538249728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760688.359053:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538249856:0@lo:13:osp-pre-0-1.0 00000001:00080000:0.0:1789760688.443092:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971197 is committed 00000001:00080000:0.0:1789760688.518580:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971198 is committed 00000001:00080000:1.0:1789760688.598995:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971199 is committed 00000100:00100000:0.0:1789760688.601993:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507121024 00000100:00100000:0.0:1789760688.602006:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507121024 00000100:00100000:0.0:1789760688.602081:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669762599428 time 669839713258 state 0x0 ref 1 00000100:00100000:0.0:1789760688.602098:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669839973674 time 669839713258 state 0x1 ref 1 00000100:00100000:0.0:1789760688.602137:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669839973674 time 669917439930 state 0x0 ref 1 00000100:00100000:0.0:1789760688.602151:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507121024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760688.602241:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15892480, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x8007 00010000:00010000:0.0:1789760688.602252:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.602687:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760688.602725:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.602733:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.602827:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760688.715016:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971200 is committed 00002000:00010000:0.0:1789760688.716058:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15892480 -> 15896576 00010000:00010000:0.0:1789760688.716086:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716105:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716117:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716126:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760688.716133:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716145:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716176:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.716197:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6653 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15892480->15896575] (req 15892480->15896575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760688.726204:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507121024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 124024us (124305us total) trans 4294971200 rc 0/0 00000100:00100000:1.0:1789760688.726227:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3910 00000100:00100000:1.0:1789760688.726240:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669839973674 time 669917439930 state 0x2 ref 0 00000100:00100000:0.0:1789760688.736689:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507121152 00000100:00100000:0.0:1789760688.736741:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507121152 00000001:00080000:1.0:1789760688.804924:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971201 is committed 00002000:00010000:0.0:1789760688.805037:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15896576 -> 15900672 00010000:00010000:0.0:1789760688.805053:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805065:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805074:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805081:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760688.805085:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805093:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805113:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.805121:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b665a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15896576->15900671] (req 15896576->15900671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.805261:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507121152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68095us (71621us total) trans 4294971201 rc 0/0 00000100:00100000:0.0:1789760688.805273:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3911 00000100:00100000:0.0:1789760688.805286:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 669917700346 time 670052449919 state 0x2 ref 0 00000100:00100000:1.0:1789760688.863153:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538249984 00000100:00100000:1.0:1789760688.863161:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538249984 00000100:00100000:1.0:1789760688.863234:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:1.0:1789760688.863255:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538249984:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760688.863295:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760688.863348:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538249984:12345-0@lo:41:osp-pre-1-0.0 Request processed in 96us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760688.863356:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00002000:00010000:0.0:1789760688.884469:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15900672 -> 15904768 00010000:00010000:0.0:1789760688.884488:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884503:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884513:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884521:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760688.884526:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884536:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884561:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.884570:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6661 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15900672->15904767] (req 15900672->15904767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.885085:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507121280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65813us (66491us total) trans 4294971202 rc 0/0 00000100:00100000:0.0:1789760688.885100:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3912 00000100:00100000:0.0:1789760688.885113:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670052710335 time 670134552781 state 0x2 ref 0 00000100:00100000:1.0:1789760688.890133:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507121408 00000100:00100000:1.0:1789760688.890147:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507121408 00000100:00100000:0.0:1789760688.891215:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670052710335 time 670134552781 state 0x0 ref 1 00000100:00100000:0.0:1789760688.891239:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670134813197 time 670134552781 state 0x1 ref 1 00000100:00100000:0.0:1789760688.891279:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670134813197 time 670206582323 state 0x0 ref 1 00000100:00100000:0.0:1789760688.891295:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507121408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760688.891365:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15904768, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x8007 00010000:00010000:0.0:1789760688.891376:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.891397:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760688.891403:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.891410:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.891430:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.034829:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15904768 -> 15908864 00010000:00010000:0.0:1789760689.034851:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034867:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034877:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034886:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.034892:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034901:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034926:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.034934:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6668 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15904768->15908863] (req 15904768->15908863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.035071:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507121408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 143771us (145622us total) trans 4294971203 rc 0/0 00000100:00100000:0.0:1789760689.035084:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3913 00000100:00100000:0.0:1789760689.035097:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670134813197 time 670206582323 state 0x2 ref 0 00000100:00100000:0.0:1789760689.039475:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00080000:0.0:1789760689.042405:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760689.062897:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760689.119635:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15908864 -> 15912960 00002000:00010000:0.0:1789760689.119641:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760687 -> 1789760688 00002000:00010000:0.0:1789760689.119643:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760687 -> 1789760688 00010000:00010000:0.0:1789760689.119661:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119676:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119686:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119738:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.119744:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119754:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119782:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.119792:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b666f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15908864->15912959] (req 15908864->15912959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.119911:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507121536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 80057us (80443us total) trans 4294971204 rc 0/0 00000100:00100000:0.0:1789760689.119923:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3914 00000100:00100000:0.0:1789760689.119934:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670206842739 time 670355138109 state 0x2 ref 0 00000100:00100000:0.0:1789760689.123999:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507121664 00000100:00100000:0.0:1789760689.124012:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507121664 00000100:00100000:0.0:1789760689.124098:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670206842739 time 670355138109 state 0x0 ref 1 00000100:00100000:0.0:1789760689.124121:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670355398525 time 670355138109 state 0x1 ref 1 00000100:00100000:0.0:1789760689.124164:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670355398525 time 670439466749 state 0x0 ref 1 00000100:00100000:0.0:1789760689.124189:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507121664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.124265:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15912960, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.124275:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6676 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.124296:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760689.124301:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.124310:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15912960->15917055] (req 15912960->15917055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.124327:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15912960->15917055] (req 15912960->15917055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.197047:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507121792 00000100:00100000:0.0:1789760689.197060:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507121792 00000100:00100000:0.0:1789760689.197129:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670355398525 time 670439466749 state 0x0 ref 1 00000100:00100000:0.0:1789760689.197147:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670439727165 time 670439466749 state 0x1 ref 1 00000100:00100000:0.0:1789760689.197171:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670439727165 time 670512475082 state 0x0 ref 1 00000100:00100000:0.0:1789760689.197199:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507121792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.197275:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15917056, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.197285:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.197304:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760689.197309:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.197316:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.197332:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.294433:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15917056 -> 15921152 00010000:00010000:0.0:1789760689.294458:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294481:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294491:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294498:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.294503:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294513:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294537:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.294554:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b667d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15917056->15921151] (req 15917056->15921151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.294730:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507121792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 97489us (98055us total) trans 4294971206 rc 0/0 00000100:00100000:0.0:1789760689.294742:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3916 00000100:00100000:0.0:1789760689.294754:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670439727165 time 670512475082 state 0x2 ref 0 00000100:00100000:0.0:1789760689.302894:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760689.375070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971207 is committed 00002000:00010000:0.0:1789760689.376193:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15921152 -> 15925248 00010000:00010000:0.0:1789760689.376212:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376236:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376246:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376254:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.376259:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376269:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376294:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.376302:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6684 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15921152->15925247] (req 15921152->15925247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.376400:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507121920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73165us (73529us total) trans 4294971207 rc 0/0 00000100:00100000:0.0:1789760689.376411:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3917 00000100:00100000:0.0:1789760689.376422:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670512735498 time 670618513179 state 0x2 ref 0 00000001:00080000:1.0:1789760689.396923:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971208 is committed 00002000:00010000:0.0:1789760689.397077:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15925248 -> 15929344 00010000:00010000:0.0:1789760689.397097:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397113:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397123:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397130:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.397136:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397146:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397171:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.397193:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b668b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15925248->15929343] (req 15925248->15929343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.397326:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507122048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 17780us (18042us total) trans 4294971208 rc 0/0 00000100:00100000:0.0:1789760689.397339:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3918 00000100:00100000:0.0:1789760689.397352:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670618773595 time 670694828247 state 0x2 ref 0 00000100:00100000:1.0:1789760689.403199:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760689.404912:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122176 00000100:00100000:0.0:1789760689.404933:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122176 00000100:00100000:0.0:1789760689.405019:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670618773595 time 670694828247 state 0x0 ref 1 00000100:00100000:0.0:1789760689.405040:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670695088663 time 670694828247 state 0x1 ref 1 00000100:00100000:0.0:1789760689.405070:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670695088663 time 670720372594 state 0x0 ref 1 00000100:00100000:0.0:1789760689.405084:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507122176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.405150:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15929344, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.405161:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.405195:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760689.405202:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.405210:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.405227:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.461515:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15929344 -> 15933440 00010000:00010000:0.0:1789760689.461536:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461820:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461832:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461841:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.461847:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461858:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461885:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.461895:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6692 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15929344->15933439] (req 15929344->15933439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.462027:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507122176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56938us (58847us total) trans 4294971209 rc 0/0 00000100:00100000:0.0:1789760689.462039:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3919 00000100:00100000:0.0:1789760689.462051:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670695088663 time 670720372594 state 0x2 ref 0 00000100:00100000:1.0:1789760689.468515:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760689.469329:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122304 00000100:00100000:0.0:1789760689.469346:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122304 00000100:00100000:1.0:1789760689.469838:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670695088663 time 670720372594 state 0x0 ref 1 00000100:00100000:1.0:1789760689.469862:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670720633010 time 670720372594 state 0x1 ref 1 00000100:00100000:1.0:1789760689.469901:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670720633010 time 670785203877 state 0x0 ref 1 00000100:00100000:1.0:1789760689.469918:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760689.469987:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15933440, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:1.0:1789760689.469997:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.470019:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:1.0:1789760689.470025:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760689.470033:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.470051:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760689.543078:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15933440 -> 15937536 00010000:00010000:1.0:1789760689.543099:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543115:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543125:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543134:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760689.543139:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543150:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543173:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.543192:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b6699 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15933440->15937535] (req 15933440->15937535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760689.543323:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73403us (74879us total) trans 4294971210 rc 0/0 00000100:00100000:1.0:1789760689.543335:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3920 00000100:00100000:1.0:1789760689.543348:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670720633010 time 670785203877 state 0x2 ref 0 00000100:00100000:0.0:1789760689.547852:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122432 00000100:00100000:0.0:1789760689.547867:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122432 00000100:00100000:0.0:1789760689.547949:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670720633010 time 670785203877 state 0x0 ref 1 00000100:00100000:0.0:1789760689.547969:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670785464293 time 670785203877 state 0x1 ref 1 00000100:00100000:0.0:1789760689.548044:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670785464293 time 670863345646 state 0x0 ref 1 00000100:00100000:0.0:1789760689.548060:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.548129:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15937536, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.548140:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.548161:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760689.548167:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.548191:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.548209:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.632427:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15937536 -> 15941632 00010000:00010000:0.0:1789760689.632447:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632460:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632475:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632483:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.632487:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632496:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632514:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.632521:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15937536->15941631] (req 15937536->15941631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.632783:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 84720us (85015us total) trans 4294971211 rc 0/0 00000100:00100000:0.0:1789760689.632794:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3921 00000100:00100000:0.0:1789760689.632803:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670785464293 time 670863345646 state 0x2 ref 0 00000100:00100000:0.0:1789760689.635418:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122560 00000100:00100000:0.0:1789760689.635432:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122560 00000100:00100000:0.0:1789760689.635517:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670785464293 time 670863345646 state 0x0 ref 1 00000100:00100000:0.0:1789760689.635536:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670863606062 time 670863345646 state 0x1 ref 1 00000100:00100000:0.0:1789760689.635572:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670863606062 time 670950875046 state 0x0 ref 1 00000100:00100000:0.0:1789760689.635588:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.635667:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15941632, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.635678:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.635742:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760689.635749:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.635757:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.635777:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.679955:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15941632 -> 15945728 00010000:00010000:0.0:1789760689.679976:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.679990:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.679999:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.680006:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.680012:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.680021:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.680040:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.680047:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b66a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15941632->15945727] (req 15941632->15945727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.680337:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507122560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44635us (44932us total) trans 4294971212 rc 0/0 00000100:00100000:0.0:1789760689.680350:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3922 00000100:00100000:0.0:1789760689.680361:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670863606062 time 670950875046 state 0x2 ref 0 00000100:00100000:1.0:1789760689.698305:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122688 00000100:00100000:1.0:1789760689.698316:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122688 00000100:00100000:1.0:1789760689.698372:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670863606062 time 670950875046 state 0x0 ref 1 00000100:00100000:1.0:1789760689.698388:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670951135462 time 670950875046 state 0x1 ref 1 00000100:00100000:1.0:1789760689.698422:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670951135462 time 671013725799 state 0x0 ref 1 00000100:00100000:1.0:1789760689.698437:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760689.698485:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15945728, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:1.0:1789760689.698493:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b66ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.698508:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760689.698512:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760689.698518:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b66ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15945728->15949823] (req 15945728->15949823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.698534:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b66ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15945728->15949823] (req 15945728->15949823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760689.721845:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971213 is committed 00000100:00100000:0.0:1789760689.738085:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507122816 00000100:00100000:0.0:1789760689.738097:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507122816 00000100:00100000:0.0:1789760689.738152:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 670951135462 time 671013725799 state 0x0 ref 1 00000100:00100000:0.0:1789760689.738167:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671013986215 time 671013725799 state 0x1 ref 1 00000100:00100000:0.0:1789760689.738219:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671013986215 time 671053515115 state 0x0 ref 1 00000100:00100000:0.0:1789760689.738233:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507122816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.738285:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15949824, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.738293:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.738309:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760689.738313:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.738319:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.738333:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.817446:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15949824 -> 15953920 00010000:00010000:0.0:1789760689.817465:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817478:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817488:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817495:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.817500:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817509:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817528:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.817535:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15949824->15953919] (req 15949824->15953919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.817638:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507122816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79402us (79889us total) trans 4294971214 rc 0/0 00000100:00100000:0.0:1789760689.817648:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3924 00000100:00100000:0.0:1789760689.817657:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671013986215 time 671053515115 state 0x2 ref 0 00000001:00080000:0.0:1789760689.817729:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971214 is committed 00000100:00100000:0.0:1789760689.823574:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760689.952194:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15958016 -> 15962112 00010000:00010000:0.0:1789760689.952220:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952234:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952242:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952249:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760689.952254:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952264:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952283:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.952291:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b66c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15958016->15962111] (req 15958016->15962111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.952383:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507123072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41410us (42059us total) trans 4294971216 rc 0/0 00000100:00100000:0.0:1789760689.952392:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3926 00000100:00100000:0.0:1789760689.952401:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671139369037 time 671226258458 state 0x2 ref 0 00000100:00100000:0.0:1789760689.963442:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507123200 00000100:00100000:0.0:1789760689.963452:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507123200 00000100:00100000:0.0:1789760689.963505:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671139369037 time 671226258458 state 0x0 ref 1 00000100:00100000:0.0:1789760689.963520:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671226518874 time 671226258458 state 0x1 ref 1 00000100:00100000:0.0:1789760689.963553:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671226518874 time 671278855931 state 0x0 ref 1 00000100:00100000:0.0:1789760689.963566:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507123200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760689.963613:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15962112, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x8007 00010000:00010000:0.0:1789760689.963621:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.963638:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760689.963642:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.963648:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.963661:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760690.086724:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971217 is committed 00002000:00010000:0.0:1789760690.086846:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15962112 -> 15966208 00010000:00010000:0.0:1789760690.086862:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086875:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086884:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086891:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760690.086896:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086905:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086924:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.086932:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15962112->15966207] (req 15962112->15966207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.087025:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507123200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 123457us (123695us total) trans 4294971217 rc 0/0 00000100:00100000:0.0:1789760690.087034:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3927 00000100:00100000:0.0:1789760690.087043:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671226518874 time 671278855931 state 0x2 ref 0 00000100:00100000:1.0:1789760690.101056:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760690.151009:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971218 is committed 00002000:00010000:0.0:1789760690.151090:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15966208 -> 15970304 00002000:00010000:0.0:1789760690.151095:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760688 -> 1789760689 00002000:00010000:0.0:1789760690.151098:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760688 -> 1789760689 00010000:00010000:0.0:1789760690.151110:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151122:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151130:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151136:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760690.151140:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151148:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151166:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.151185:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b66d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15966208->15970303] (req 15966208->15970303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.151371:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507123328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50096us (50319us total) trans 4294971218 rc 0/0 00000100:00100000:0.0:1789760690.151382:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3928 00000100:00100000:0.0:1789760690.151391:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671279116347 time 671416562784 state 0x2 ref 0 00000100:00100000:1.0:1789760690.159332:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760690.160237:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507123456 00000100:00100000:0.0:1789760690.160248:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507123456 00000100:00100000:0.0:1789760690.160310:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671279116347 time 671416562784 state 0x0 ref 1 00000100:00100000:0.0:1789760690.160327:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671416823200 time 671416562784 state 0x1 ref 1 00000100:00100000:0.0:1789760690.160350:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671416823200 time 671475653708 state 0x0 ref 1 00000100:00100000:0.0:1789760690.160362:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507123456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760690.160413:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15970304, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x8007 00010000:00010000:0.0:1789760690.160422:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.160439:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760690.160444:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.160450:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15970304->15974399] (req 15970304->15974399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.160463:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b66d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15970304->15974399] (req 15970304->15974399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760690.220065:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971219 is committed 00000100:00100000:0.0:1789760690.225371:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507123584 00000100:00100000:0.0:1789760690.225381:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507123584 00000100:00100000:0.0:1789760690.225436:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671416823200 time 671475653708 state 0x0 ref 1 00000100:00100000:0.0:1789760690.225452:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671475914124 time 671475653708 state 0x1 ref 1 00000100:00100000:0.0:1789760690.225827:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671475914124 time 671541128749 state 0x0 ref 1 00000100:00100000:0.0:1789760690.225842:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507123584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760690.225889:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15974400, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x8007 00010000:00010000:0.0:1789760690.225898:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b66df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.225911:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760690.225915:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.225921:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b66df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15974400->15978495] (req 15974400->15978495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.225934:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b66df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15974400->15978495] (req 15974400->15978495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760690.299202:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760690.299339:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507123712 00000100:00100000:0.0:1789760690.299349:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507123712 00000100:00100000:0.0:1789760690.299404:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671475914124 time 671541128749 state 0x0 ref 1 00000100:00100000:0.0:1789760690.299420:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671541389165 time 671541128749 state 0x1 ref 1 00000100:00100000:0.0:1789760690.299456:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671541389165 time 671614759354 state 0x0 ref 1 00000100:00100000:0.0:1789760690.299475:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507123712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760690.299519:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 15978496, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x8007 00010000:00010000:0.0:1789760690.299527:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.299543:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760690.299547:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.299553:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.299565:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760690.378476:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 15978496 -> 15982592 00010000:00010000:0.0:1789760690.378494:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378512:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378521:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378527:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760690.378532:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378541:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378559:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.378567:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b66e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [15978496->15982591] (req 15978496->15982591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.378664:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507123712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79186us (79467us total) trans 4294971221 rc 0/0 00000100:00100000:0.0:1789760690.378674:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3931 00000100:00100000:0.0:1789760690.378684:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 671541389165 time 671614759354 state 0x2 ref 0 00000100:00100000:1.0:1789760690.383794:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760690.565725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971223 is committed 00000100:00100000:0.0:1789760690.576852:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760690.719925:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971224 is committed 00000001:00080000:0.0:1789760690.866079:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971225 is committed 00000001:00080000:0.0:1789760691.024373:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971226 is committed 00000100:00100000:0.0:1789760691.048343:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00080000:0.0:1789760691.049591:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760691.100686:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760691.477732:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971229 is committed 00000100:00100000:0.0:1789760691.481949:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760691.615972:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16015360 -> 16019456 00010000:00010000:0.0:1789760691.615999:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616015:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616026:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616035:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760691.616041:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616055:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616079:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.616088:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6725 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16015360->16019455] (req 16015360->16019455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.616232:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507124864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 134031us (134291us total) trans 4294971230 rc 0/0 00000100:00100000:0.0:1789760691.616244:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3940 00000100:00100000:0.0:1789760691.616257:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672706090629 time 672797485430 state 0x2 ref 0 00000100:00100000:0.0:1789760691.619759:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507124992 00000100:00100000:0.0:1789760691.619773:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507124992 00000100:00100000:0.0:1789760691.619854:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672706090629 time 672797485430 state 0x0 ref 1 00000100:00100000:0.0:1789760691.619888:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672797745846 time 672797485430 state 0x1 ref 1 00000100:00100000:0.0:1789760691.619915:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672797745846 time 672935218636 state 0x0 ref 1 00000100:00100000:0.0:1789760691.619928:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507124992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760691.619994:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16019456, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x8007 00010000:00010000:0.0:1789760691.620007:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.620025:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760691.620030:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.620037:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.620053:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.621435:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971230 is committed 00002000:00010000:0.0:1789760691.676512:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16019456 -> 16023552 00010000:00010000:0.0:1789760691.676542:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676559:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676570:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676578:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760691.676584:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676595:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676797:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.676808:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b672c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16019456->16023551] (req 16019456->16023551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.676993:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971231 is committed 00000100:00100000:0.0:1789760691.677058:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507124992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57124us (57485us total) trans 4294971231 rc 0/0 00000100:00100000:0.0:1789760691.677075:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3941 00000100:00100000:0.0:1789760691.677087:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672797745846 time 672935218636 state 0x2 ref 0 00000100:00100000:1.0:1789760691.691840:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760691.762137:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16023552 -> 16027648 00010000:00010000:0.0:1789760691.762171:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762187:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762205:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762214:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760691.762219:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762229:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762254:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762263:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6733 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16023552->16027647] (req 16023552->16027647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.762541:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507125120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70166us (70711us total) trans 4294971232 rc 0/0 00000100:00100000:0.0:1789760691.762554:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3942 00000100:00100000:0.0:1789760691.762567:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672935479052 time 673007656072 state 0x2 ref 0 00000100:00100000:0.0:1789760691.768463:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507125248 00000100:00100000:0.0:1789760691.768477:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507125248 00000100:00100000:0.0:1789760691.768546:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 672935479052 time 673007656072 state 0x0 ref 1 00000100:00100000:0.0:1789760691.768564:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673007916488 time 673007656072 state 0x1 ref 1 00000100:00100000:0.0:1789760691.768598:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673007916488 time 673083901771 state 0x0 ref 1 00000100:00100000:0.0:1789760691.768621:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760691.768684:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16027648, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x8007 00010000:00010000:0.0:1789760691.768721:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.768741:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760691.768746:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.768753:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.768768:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.837919:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16027648 -> 16031744 00010000:00010000:0.0:1789760691.837939:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.837954:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.837964:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.837972:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760691.837978:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.837989:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.838131:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.838141:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b673a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16027648->16031743] (req 16027648->16031743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.838374:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69747us (70046us total) trans 4294971233 rc 0/0 00000100:00100000:0.0:1789760691.838390:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3943 00000100:00100000:0.0:1789760691.838402:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673007916488 time 673083901771 state 0x2 ref 0 00000100:00100000:0.0:1789760691.842859:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507125376 00000100:00100000:0.0:1789760691.843276:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507125376 00000100:00100000:0.0:1789760691.843353:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673007916488 time 673083901771 state 0x0 ref 1 00000100:00100000:0.0:1789760691.843372:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673084162187 time 673083901771 state 0x1 ref 1 00000100:00100000:0.0:1789760691.843409:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673084162187 time 673158710934 state 0x0 ref 1 00000100:00100000:0.0:1789760691.843424:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507125376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760691.843491:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16031744, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x8007 00010000:00010000:0.0:1789760691.843503:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6741 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.843523:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760691.843528:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.843536:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6741 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16031744->16035839] (req 16031744->16035839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.843554:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6741 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16031744->16035839] (req 16031744->16035839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.921976:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507125504 00000100:00100000:0.0:1789760691.921985:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507125504 00000100:00100000:0.0:1789760691.922037:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673084162187 time 673158710934 state 0x0 ref 1 00000100:00100000:0.0:1789760691.922053:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673158971350 time 673158710934 state 0x1 ref 1 00000100:00100000:0.0:1789760691.922120:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673158971350 time 673237422660 state 0x0 ref 1 00000100:00100000:0.0:1789760691.922133:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760691.922190:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16035840, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x8007 00010000:00010000:0.0:1789760691.922199:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.922214:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760691.922219:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.922226:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.922239:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.932623:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760691.932666:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760691.932719:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760691.932741:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760691.932775:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760691.932802:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507125888 00000100:00100000:0.0:1789760691.932811:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507125888 00000100:00100000:0.0:1789760691.932861:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507126144 00000100:00100000:0.0:1789760691.932865:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507126144 00000100:00100000:0.0:1789760691.932879:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507125760 00000100:00100000:0.0:1789760691.932883:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507125760 00000100:00100000:0.0:1789760691.932895:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507126016 00000100:00100000:0.0:1789760691.932897:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507126016 00000100:00100000:0.0:1789760691.932913:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 788 00000100:00100000:0.0:1789760691.932922:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125888:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.932989:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125888:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 68us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.932997:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 788 00000100:00100000:0.0:1789760691.933014:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 789 00000100:00100000:0.0:1789760691.933020:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507126144:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.933045:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507126144:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.933052:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 789 00000100:00100000:0.0:1789760691.933063:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 790 00000100:00100000:0.0:1789760691.933069:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507125760:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.933091:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507125760:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.933098:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 790 00000100:00100000:0.0:1789760691.933108:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 791 00000100:00100000:0.0:1789760691.933114:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507126016:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.933137:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507126016:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.933143:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 791 00000100:00080000:0.0:1789760691.933176:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760691.933180:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00010000:0.0:1789760691.973044:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16035840 -> 16039936 00010000:00010000:0.0:1789760691.973058:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973071:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973081:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973088:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760691.973092:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973101:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973118:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.973125:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6748 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16035840->16039935] (req 16035840->16039935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.973271:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507125504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51128us (51393us total) trans 4294971235 rc 0/0 00000100:00100000:0.0:1789760691.973282:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3945 00000100:00100000:0.0:1789760691.973292:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673158971350 time 673237422660 state 0x2 ref 0 00000100:00100000:0.0:1789760691.977965:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760692.022845:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16039936 -> 16044032 00010000:00010000:0.0:1789760692.022860:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022872:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022882:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022891:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.022896:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022905:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022921:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.022926:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b674f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16039936->16044031] (req 16039936->16044031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.023052:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507126272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44818us (45090us total) trans 4294971236 rc 0/0 00000100:00100000:0.0:1789760692.023063:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3946 00000100:00100000:0.0:1789760692.023073:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673237683076 time 673293519043 state 0x2 ref 0 00000100:00100000:0.0:1789760692.027913:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760692.122323:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16044032 -> 16048128 00002000:00010000:0.0:1789760692.122332:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760690 -> 1789760691 00002000:00010000:0.0:1789760692.122334:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760690 -> 1789760691 00010000:00010000:0.0:1789760692.122348:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122361:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122369:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122377:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.122382:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122403:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122423:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.122436:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6756 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16044032->16048127] (req 16044032->16048127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.122545:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507126400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 94088us (94637us total) trans 4294971237 rc 0/0 00000100:00100000:0.0:1789760692.122562:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3947 00000100:00100000:0.0:1789760692.122573:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673293779459 time 673343679924 state 0x2 ref 0 00000100:00100000:0.0:1789760692.133061:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507126528 00000100:00100000:0.0:1789760692.133070:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507126528 00000100:00100000:0.0:1789760692.133134:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673293779459 time 673343679924 state 0x0 ref 1 00000100:00100000:0.0:1789760692.133150:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673343940340 time 673343679924 state 0x1 ref 1 00000100:00100000:0.0:1789760692.133200:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673343940340 time 673448493293 state 0x0 ref 1 00000100:00100000:0.0:1789760692.133214:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507126528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.133265:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16048128, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.133274:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.133287:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760692.133292:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.133298:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.133319:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.219070:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16048128 -> 16052224 00010000:00010000:0.0:1789760692.219084:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219096:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219103:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219110:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.219114:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219123:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219141:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.219149:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b675d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16048128->16052223] (req 16048128->16052223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.219259:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507126528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 86042us (86412us total) trans 4294971238 rc 0/0 00000100:00100000:0.0:1789760692.219459:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3948 00000100:00100000:0.0:1789760692.219470:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673343940340 time 673448493293 state 0x2 ref 0 00000100:00100000:1.0:1789760692.241472:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760692.241737:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507126656 00000100:00100000:0.0:1789760692.241747:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507126656 00000100:00100000:0.0:1789760692.241804:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673343940340 time 673448493293 state 0x0 ref 1 00000100:00100000:0.0:1789760692.241821:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673448753709 time 673448493293 state 0x1 ref 1 00000100:00100000:0.0:1789760692.241901:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673448753709 time 673557203885 state 0x0 ref 1 00000100:00100000:0.0:1789760692.241915:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507126656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.241976:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16052224, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.241986:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.242003:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760692.242008:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.242016:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.242031:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.317635:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16052224 -> 16056320 00010000:00010000:0.0:1789760692.317650:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317661:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317672:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317679:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.317684:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317712:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317734:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.317743:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6764 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16052224->16056319] (req 16052224->16056319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.317840:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507126656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75922us (76374us total) trans 4294971239 rc 0/0 00000100:00100000:0.0:1789760692.317856:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3949 00000100:00100000:0.0:1789760692.317866:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673448753709 time 673557203885 state 0x2 ref 0 00000100:00100000:0.0:1789760692.322286:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760692.364844:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507126912 00000100:00100000:0.0:1789760692.364853:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507126912 00000100:00100000:0.0:1789760692.364905:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673557464301 time 673637932044 state 0x0 ref 1 00000100:00100000:0.0:1789760692.364921:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673638192460 time 673637932044 state 0x1 ref 1 00000100:00100000:0.0:1789760692.364951:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673638192460 time 673680254162 state 0x0 ref 1 00000100:00100000:0.0:1789760692.364965:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507126912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.365014:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16060416, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.365023:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.365038:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760692.365043:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.365049:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.365064:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760692.441741:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971241 is committed 00002000:00010000:0.0:1789760692.442091:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16060416 -> 16064512 00010000:00010000:0.0:1789760692.442108:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442121:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442131:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442140:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.442145:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442165:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442187:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.442207:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6772 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16060416->16064511] (req 16060416->16064511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.442539:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507126912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77571us (77778us total) trans 4294971241 rc 0/0 00000100:00100000:0.0:1789760692.442550:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3951 00000100:00100000:0.0:1789760692.442559:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673638192460 time 673680254162 state 0x2 ref 0 00000100:00100000:1.0:1789760692.448527:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538250240:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448593:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448649:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b45a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538250752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448682:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448733:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538251264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448759:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448779:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538251776:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448800:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448822:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1789760692.448827:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538250368:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448853:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448870:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538250880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448888:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448904:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538251392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.448939:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.448969:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538250240 00000100:00100000:1.0:1789760692.448977:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538250240 00000100:00100000:1.0:1789760692.449019:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1789760692.449028:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538250240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.449077:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538250240:12345-0@lo:400:kworker.0 Request processed in 49us (484us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760692.449085:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1789760692.449117:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538250752 00000100:00100000:1.0:1789760692.449121:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538250752 00000100:00100000:1.0:1789760692.449147:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1789760692.449166:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22dc7e850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538250752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.449198:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22dc7e850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538250752:12345-0@lo:400:kworker.0 Request processed in 45us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760692.449205:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1789760692.450859:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538250368:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450870:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538250880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450878:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538251392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450887:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538250240:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450896:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b45a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538250752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450904:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538251264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.450995:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538251776:0@lo:400:kworker.0 00000100:00080000:1.0:1789760692.451041:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760692.451045:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760692.451048:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760692.451058:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538250624:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.451089:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.451116:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538251136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.451134:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.451149:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538251648:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.451177:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.451298:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538250496:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451360:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.451379:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538251008:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451408:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.451422:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538251520:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451438:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.451461:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538251136 00000100:00100000:0.0:1789760692.451465:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538251136 00000100:00100000:0.0:1789760692.451492:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538251648 00000100:00100000:0.0:1789760692.451494:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538251648 00000100:00100000:0.0:1789760692.451507:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1789760692.451512:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538251136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451542:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538251136:12345-0@lo:400:kworker.0 Request processed in 29us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.451547:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1789760692.451558:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1789760692.451563:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538251648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451581:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538251648:12345-0@lo:400:kworker.0 Request processed in 19us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.451587:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1789760692.451612:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538250624 00000100:00100000:0.0:1789760692.451615:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538250624 00000100:00100000:0.0:1789760692.451631:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538251008 00000100:00100000:0.0:1789760692.451633:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538251008 00000100:00100000:0.0:1789760692.451645:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538251520 00000100:00100000:0.0:1789760692.451647:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538251520 00000100:00100000:0.0:1789760692.451661:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 795 00000100:00100000:0.0:1789760692.451666:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538250624:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451690:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538250624:12345-0@lo:400:kworker.0 Request processed in 24us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.451722:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 795 00000100:00100000:0.0:1789760692.451733:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 796 00000100:00100000:0.0:1789760692.451739:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538251008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451766:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538251008:12345-0@lo:400:kworker.0 Request processed in 27us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.451771:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 796 00000100:00100000:0.0:1789760692.451780:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 797 00000100:00100000:0.0:1789760692.451785:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538251520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451803:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538251520:12345-0@lo:400:kworker.0 Request processed in 19us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.451808:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 797 00000100:00100000:0.0:1789760692.451833:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538251008:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.451841:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538251520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.452357:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538250496 00000100:00100000:1.0:1789760692.452363:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538250496 00000100:00100000:1.0:1789760692.452459:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:1.0:1789760692.452467:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538250496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.452516:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538250496:12345-0@lo:400:kworker.0 Request processed in 49us (1154us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760692.452522:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00080000:1.0:1789760692.452540:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760692.452544:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760692.452546:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760692.452567:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538250624:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.452577:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538251136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.452586:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538251648:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.452847:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538250496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.456968:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760692.456991:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538251904:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760692.457061:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760692.457110:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538251904 00000100:00100000:1.0:1789760692.457117:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538251904 00000100:00100000:1.0:1789760692.457172:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789760692.457183:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538251904:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760692.457572:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760692.457641:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538251904:12345-0@lo:13:osp-pre-1-0.0 Request processed in 459us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760692.457649:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789760692.459022:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760692.459054:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127040 00000100:00100000:1.0:1789760692.459061:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127040 00000100:00100000:1.0:1789760692.459106:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673638192460 time 673680254162 state 0x0 ref 1 00000100:00100000:1.0:1789760692.459121:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673680514578 time 673680254162 state 0x1 ref 1 00000100:00100000:1.0:1789760692.459145:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673680514578 time 673774448213 state 0x0 ref 1 00000100:00100000:1.0:1789760692.459169:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507127040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760692.459233:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16064512, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:1.0:1789760692.459243:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.459258:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:1.0:1789760692.459263:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.459269:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.459284:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.467824:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538252032:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760692.467894:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.467962:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538252032 00000100:00100000:0.0:1789760692.467971:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538252032 00000100:00100000:0.0:1789760692.468019:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:0.0:1789760692.468029:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538252032:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760692.468052:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760692.468096:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538252032:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.468104:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:0.0:1789760692.468146:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538252032:0@lo:13:osp-pre-1-1.0 00000001:00080000:1.0:1789760692.565118:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971242 is committed 00002000:00010000:0.0:1789760692.565380:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16064512 -> 16068608 00010000:00010000:0.0:1789760692.565406:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565419:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565428:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565437:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.565442:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565453:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565475:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.565484:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6779 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16064512->16068607] (req 16064512->16068607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.565578:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507127040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 106418us (106557us total) trans 4294971242 rc 0/0 00000100:00100000:0.0:1789760692.565590:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3952 00000100:00100000:0.0:1789760692.565919:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673680514578 time 673774448213 state 0x2 ref 0 00000100:00100000:1.0:1789760692.575833:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760692.669900:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971243 is committed 00002000:00010000:0.0:1789760692.669987:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16068608 -> 16072704 00010000:00010000:0.0:1789760692.670002:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670013:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670021:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670028:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.670033:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670042:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670062:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.670069:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6780 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16068608->16072703] (req 16068608->16072703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.670212:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507127168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 94020us (94383us total) trans 4294971243 rc 0/0 00000100:00100000:0.0:1789760692.670224:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3953 00000100:00100000:0.0:1789760692.670234:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673774708629 time 673891478472 state 0x2 ref 0 00000100:00100000:0.0:1789760692.674309:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127296 00000100:00100000:0.0:1789760692.674319:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127296 00000100:00100000:0.0:1789760692.674374:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673774708629 time 673891478472 state 0x0 ref 1 00000100:00100000:0.0:1789760692.674399:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673891738888 time 673891478472 state 0x1 ref 1 00000100:00100000:0.0:1789760692.674428:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673891738888 time 673989731831 state 0x0 ref 1 00000100:00100000:0.0:1789760692.674439:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507127296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.674485:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16072704, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.674493:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.674509:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760692.674513:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.674520:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.674533:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.772194:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16072704 -> 16076800 00000001:00080000:1.0:1789760692.772505:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971244 is committed 00010000:00010000:0.0:1789760692.772676:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772690:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772719:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772726:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760692.772731:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772741:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772766:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.772774:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6787 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16072704->16076799] (req 16072704->16076799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.772871:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507127296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 98426us (98624us total) trans 4294971244 rc 0/0 00000100:00100000:0.0:1789760692.772886:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3954 00000100:00100000:0.0:1789760692.772895:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673891738888 time 673989731831 state 0x2 ref 0 00000100:00100000:0.0:1789760692.791956:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760692.792167:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127424 00000100:00100000:1.0:1789760692.792177:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127424 00000100:00100000:1.0:1789760692.792349:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673891738888 time 673989731831 state 0x0 ref 1 00000100:00100000:1.0:1789760692.792364:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673989992247 time 673989731831 state 0x1 ref 1 00000100:00100000:1.0:1789760692.792487:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673989992247 time 674107789058 state 0x0 ref 1 00000100:00100000:1.0:1789760692.792500:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507127424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760692.792544:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16076800, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:1.0:1789760692.792553:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.792567:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760692.792572:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.792577:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.792590:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760692.880832:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16076800 -> 16080896 00010000:00010000:1.0:1789760692.880849:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880862:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880871:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880884:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760692.880889:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880898:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880931:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.880939:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b678e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16076800->16080895] (req 16076800->16080895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.881027:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507127424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 88525us (89076us total) trans 4294971245 rc 0/0 00000100:00100000:1.0:1789760692.881037:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3955 00000100:00100000:1.0:1789760692.881048:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673989992247 time 674107789058 state 0x2 ref 0 00000100:00100000:1.0:1789760692.889182:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760692.889391:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127552 00000100:00100000:0.0:1789760692.889403:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127552 00000100:00100000:0.0:1789760692.889457:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 673989992247 time 674107789058 state 0x0 ref 1 00000100:00100000:0.0:1789760692.889473:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674108049474 time 674107789058 state 0x1 ref 1 00000100:00100000:0.0:1789760692.889502:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674108049474 time 674204806083 state 0x0 ref 1 00000100:00100000:0.0:1789760692.889515:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507127552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.889563:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16080896, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.889572:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.889588:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760692.889592:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.889598:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.889612:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760692.942122:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16080896 -> 16084992 00010000:00010000:1.0:1789760692.942142:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942165:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942181:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942196:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760692.942201:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942218:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942237:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.942246:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6795 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16080896->16084991] (req 16080896->16084991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.942332:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507127552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52814us (53155us total) trans 4294971246 rc 0/0 00000100:00100000:1.0:1789760692.942341:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3956 00000100:00100000:1.0:1789760692.942351:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674108049474 time 674204806083 state 0x2 ref 0 00000100:00100000:0.0:1789760692.950454:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127680 00000100:00100000:0.0:1789760692.950465:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127680 00000100:00100000:0.0:1789760692.950522:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674108049474 time 674204806083 state 0x0 ref 1 00000100:00100000:0.0:1789760692.950537:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674205066499 time 674204806083 state 0x1 ref 1 00000100:00100000:0.0:1789760692.950567:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674205066499 time 674265869991 state 0x0 ref 1 00000100:00100000:0.0:1789760692.950578:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507127680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760692.950627:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16084992, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:0.0:1789760692.950636:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b679c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.950653:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760692.950657:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.950664:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b679c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16084992->16089087] (req 16084992->16089087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.950679:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b679c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16084992->16089087] (req 16084992->16089087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760693.037141:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971247 is committed 00000100:00100000:1.0:1789760693.041316:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127808 00000100:00100000:1.0:1789760693.041326:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127808 00000100:00100000:1.0:1789760693.041404:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674205066499 time 674265869991 state 0x0 ref 1 00000100:00100000:1.0:1789760693.041422:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674266130407 time 674265869991 state 0x1 ref 1 00000100:00100000:1.0:1789760693.041454:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674266130407 time 674356757458 state 0x0 ref 1 00000100:00100000:1.0:1789760693.041467:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507127808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.041515:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16089088, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x8007 00010000:00010000:1.0:1789760693.041524:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.041540:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.041545:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.041551:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.041564:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760693.080494:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971248 is committed 00002000:00010000:1.0:1789760693.080562:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16089088 -> 16093184 00002000:00010000:1.0:1789760693.080566:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760691 -> 1789760692 00002000:00010000:1.0:1789760693.080569:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760691 -> 1789760692 00010000:00010000:1.0:1789760693.080581:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081397:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081415:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081422:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.081427:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081437:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081459:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.081466:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b67a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16089088->16093183] (req 16089088->16093183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.081610:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507127808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40132us (40493us total) trans 4294971248 rc 0/0 00000100:00100000:1.0:1789760693.081620:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3958 00000100:00100000:1.0:1789760693.081631:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674266130407 time 674356757458 state 0x2 ref 0 00000100:00100000:1.0:1789760693.087092:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.087173:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507127936 00000100:00100000:0.0:1789760693.087184:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507127936 00000100:00100000:0.0:1789760693.087238:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674266130407 time 674356757458 state 0x0 ref 1 00000100:00100000:0.0:1789760693.087254:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674357017874 time 674356757458 state 0x1 ref 1 00000100:00100000:0.0:1789760693.087288:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674357017874 time 674402591095 state 0x0 ref 1 00000100:00100000:0.0:1789760693.087305:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414dd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507127936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.087351:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16093184, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.087360:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.087376:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760693.087380:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.087386:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.087399:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.117508:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16093184 -> 16097280 00010000:00010000:0.0:1789760693.117529:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117541:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117551:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117559:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760693.117563:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117574:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117594:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.117603:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16093184->16097279] (req 16093184->16097279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.117731:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414dd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507127936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30387us (30607us total) trans 4294971249 rc 0/0 00000100:00100000:0.0:1789760693.117741:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3959 00000100:00100000:0.0:1789760693.117751:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674357017874 time 674402591095 state 0x2 ref 0 00000001:00080000:0.0:1789760693.119783:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971249 is committed 00000100:00100000:0.0:1789760693.123483:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128064 00000100:00100000:0.0:1789760693.123493:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128064 00000100:00100000:0.0:1789760693.123551:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674357017874 time 674402591095 state 0x0 ref 1 00000100:00100000:0.0:1789760693.123567:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674402851511 time 674402591095 state 0x1 ref 1 00000100:00100000:0.0:1789760693.123593:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674402851511 time 674438896598 state 0x0 ref 1 00000100:00100000:0.0:1789760693.123605:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.123651:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16097280, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.123662:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.123677:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760693.123682:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.123688:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.123744:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.147799:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760693.150652:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971250 is committed 00002000:00010000:0.0:1789760693.150955:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16097280 -> 16101376 00010000:00010000:0.0:1789760693.150970:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.150980:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.150987:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.150992:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760693.150995:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.151002:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.151023:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.151031:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16097280->16101375] (req 16097280->16101375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.151119:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27509us (27833us total) trans 4294971250 rc 0/0 00000100:00100000:0.0:1789760693.151129:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3960 00000100:00100000:0.0:1789760693.151139:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674402851511 time 674438896598 state 0x2 ref 0 00000100:00100000:1.0:1789760693.154034:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128192 00000100:00100000:1.0:1789760693.154042:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128192 00000100:00100000:1.0:1789760693.154096:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674402851511 time 674438896598 state 0x0 ref 1 00000100:00100000:1.0:1789760693.154115:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674439157014 time 674438896598 state 0x1 ref 1 00000100:00100000:1.0:1789760693.154202:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674439157014 time 674469504253 state 0x0 ref 1 00000100:00100000:1.0:1789760693.154214:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507128192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.154260:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16101376, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.154269:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b67b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.154284:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.154288:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.154294:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b67b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16101376->16105471] (req 16101376->16105471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.154308:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b67b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16101376->16105471] (req 16101376->16105471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760693.186412:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971251 is committed 00000100:00100000:1.0:1789760693.190311:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.190374:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128320 00000100:00100000:0.0:1789760693.190386:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128320 00000100:00100000:0.0:1789760693.190447:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674439157014 time 674469504253 state 0x0 ref 1 00000100:00100000:0.0:1789760693.190465:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674469764669 time 674469504253 state 0x1 ref 1 00000100:00100000:0.0:1789760693.190491:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674469764669 time 674505794940 state 0x0 ref 1 00000100:00100000:0.0:1789760693.190505:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507128320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.190554:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16105472, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.190563:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.190579:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760693.190585:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.190592:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.190607:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.221157:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16105472 -> 16109568 00010000:00010000:1.0:1789760693.221176:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221198:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221214:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221221:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.221227:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221236:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221255:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.221263:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b67bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16105472->16109567] (req 16105472->16109567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.221380:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507128320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30871us (31072us total) trans 4294971252 rc 0/0 00000100:00100000:1.0:1789760693.221391:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3962 00000100:00100000:1.0:1789760693.221401:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674469764669 time 674505794940 state 0x2 ref 0 00000100:00100000:1.0:1789760693.226080:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.226171:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128448 00000100:00100000:0.0:1789760693.226182:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128448 00000100:00100000:0.0:1789760693.226248:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674469764669 time 674505794940 state 0x0 ref 1 00000100:00100000:0.0:1789760693.226264:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674506055356 time 674505794940 state 0x1 ref 1 00000100:00100000:0.0:1789760693.226295:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674506055356 time 674541597841 state 0x0 ref 1 00000100:00100000:0.0:1789760693.226307:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507128448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.226359:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16109568, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.226369:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.226386:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760693.226391:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.226399:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.226415:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.268669:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16109568 -> 16113664 00010000:00010000:1.0:1789760693.268690:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268735:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268745:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268753:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.268757:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268765:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268787:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.268795:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b67c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16109568->16113663] (req 16109568->16113663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.268881:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507128448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42570us (42806us total) trans 4294971253 rc 0/0 00000100:00100000:1.0:1789760693.268891:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3963 00000100:00100000:1.0:1789760693.268900:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674506055356 time 674541597841 state 0x2 ref 0 00000100:00100000:1.0:1789760693.274526:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128576 00000100:00100000:1.0:1789760693.274536:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128576 00000100:00100000:1.0:1789760693.274664:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674506055356 time 674541597841 state 0x0 ref 1 00000100:00100000:1.0:1789760693.274681:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674541858257 time 674541597841 state 0x1 ref 1 00000100:00100000:1.0:1789760693.274733:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674541858257 time 674590035696 state 0x0 ref 1 00000100:00100000:1.0:1789760693.274746:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.274795:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16113664, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.274805:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.274821:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.274826:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.274833:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.274847:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.332984:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16113664 -> 16117760 00010000:00010000:0.0:1789760693.333000:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333020:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333030:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333043:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760693.333047:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333056:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333082:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.333114:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b67cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16113664->16117759] (req 16113664->16117759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.333457:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58708us (59070us total) trans 4294971254 rc 0/0 00000100:00100000:0.0:1789760693.333472:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3964 00000100:00100000:0.0:1789760693.333484:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674541858257 time 674590035696 state 0x2 ref 0 00000100:00100000:1.0:1789760693.338754:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760693.382395:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971255 is committed 00000100:00100000:1.0:1789760693.389422:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.389571:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507128832 00000100:00100000:0.0:1789760693.389582:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507128832 00000100:00100000:0.0:1789760693.389637:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674590296112 time 674654262315 state 0x0 ref 1 00000100:00100000:0.0:1789760693.389652:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674654522731 time 674654262315 state 0x1 ref 1 00000100:00100000:0.0:1789760693.389689:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674654522731 time 674704992541 state 0x0 ref 1 00000100:00100000:0.0:1789760693.390280:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.390424:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16121856, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.390433:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.390448:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760693.390453:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.390459:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.390473:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.423478:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16121856 -> 16125952 00010000:00010000:1.0:1789760693.423506:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423527:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423537:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423544:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.423549:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423556:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423575:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.423583:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b67db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16121856->16125951] (req 16121856->16125951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.423940:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507128832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33661us (34522us total) trans 4294971256 rc 0/0 00000100:00100000:1.0:1789760693.423952:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3966 00000100:00100000:1.0:1789760693.423964:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674654522731 time 674704992541 state 0x2 ref 0 00000100:00100000:0.0:1789760693.427744:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760693.468105:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760693.468129:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105822a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538252160:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760693.468214:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760693.468569:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105822a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538252160:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760693.468607:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234738700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538252288:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760693.468650:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:1.0:1789760693.483544:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16125952 -> 16130048 00010000:00010000:1.0:1789760693.483566:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483580:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483588:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483595:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.483600:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483611:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483631:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.483640:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b67e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16125952->16130047] (req 16125952->16130047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.483769:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507128960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55851us (56032us total) trans 4294971257 rc 0/0 00000100:00100000:1.0:1789760693.483780:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3967 00000100:00100000:1.0:1789760693.483790:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674705252957 time 674743205395 state 0x2 ref 0 00000100:00100000:0.0:1789760693.491261:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760693.525946:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16130048 -> 16134144 00010000:00010000:1.0:1789760693.525963:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.525989:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.525998:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.526005:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.526010:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.526021:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.526040:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.526052:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b67e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16130048->16134143] (req 16130048->16134143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.526164:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507129088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34627us (34908us total) trans 4294971258 rc 0/0 00000100:00100000:1.0:1789760693.526174:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3968 00000100:00100000:1.0:1789760693.526193:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674743465811 time 674806824913 state 0x2 ref 0 00000100:00100000:1.0:1789760693.535587:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.535688:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129216 00000100:00100000:0.0:1789760693.535782:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129216 00000100:00100000:1.0:1789760693.535913:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674743465811 time 674806824913 state 0x0 ref 1 00000100:00100000:1.0:1789760693.535929:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674807085329 time 674806824913 state 0x1 ref 1 00000100:00100000:1.0:1789760693.535966:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674807085329 time 674851268449 state 0x0 ref 1 00000100:00100000:1.0:1789760693.536094:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507129216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.536158:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16134144, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.536169:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b67f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.536192:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.536197:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.536203:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b67f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16134144->16138239] (req 16134144->16138239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.536392:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b67f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16134144->16138239] (req 16134144->16138239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760693.605344:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971259 is committed 00000100:00100000:0.0:1789760693.614192:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760693.614264:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129344 00000100:00100000:1.0:1789760693.614274:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129344 00000100:00100000:1.0:1789760693.614329:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674807085329 time 674851268449 state 0x0 ref 1 00000100:00100000:1.0:1789760693.614345:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674851528865 time 674851268449 state 0x1 ref 1 00000100:00100000:1.0:1789760693.614377:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674851528865 time 674929680069 state 0x0 ref 1 00000100:00100000:1.0:1789760693.614389:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507129344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.614441:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16138240, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.614450:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.614465:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.614470:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.614476:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.614491:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.656573:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16138240 -> 16142336 00010000:00010000:0.0:1789760693.656594:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656608:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656617:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656625:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760693.656630:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656639:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656659:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.656667:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16138240->16142335] (req 16138240->16142335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.656788:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507129344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42396us (42601us total) trans 4294971260 rc 0/0 00000100:00100000:0.0:1789760693.656800:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3970 00000100:00100000:0.0:1789760693.656810:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674851528865 time 674929680069 state 0x2 ref 0 00000100:00100000:0.0:1789760693.672520:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760693.672789:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129472 00000100:00100000:1.0:1789760693.672800:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129472 00000100:00100000:1.0:1789760693.672856:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674851528865 time 674929680069 state 0x0 ref 1 00000100:00100000:1.0:1789760693.672872:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674929940485 time 674929680069 state 0x1 ref 1 00000100:00100000:1.0:1789760693.672906:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674929940485 time 674988208306 state 0x0 ref 1 00000100:00100000:1.0:1789760693.672919:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507129472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.672967:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16142336, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.672975:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.672990:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760693.672993:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.672999:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.673013:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.735855:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16142336 -> 16146432 00010000:00010000:0.0:1789760693.735871:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735884:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735894:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735902:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760693.735906:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735915:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735936:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.735944:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b67fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16142336->16146431] (req 16142336->16146431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.736039:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507129472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63118us (63526us total) trans 4294971261 rc 0/0 00000100:00100000:0.0:1789760693.736050:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3971 00000100:00100000:0.0:1789760693.736059:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674929940485 time 674988208306 state 0x2 ref 0 00000100:00100000:1.0:1789760693.747346:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760693.808527:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971262 is committed 00002000:00010000:1.0:1789760693.808758:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16146432 -> 16150528 00010000:00010000:1.0:1789760693.808773:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808787:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808802:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808810:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.808815:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808823:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808842:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.808851:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6805 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16146432->16150527] (req 16146432->16150527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.808938:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507129600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61386us (61598us total) trans 4294971262 rc 0/0 00000100:00100000:1.0:1789760693.808948:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3972 00000100:00100000:1.0:1789760693.808957:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674988468722 time 675062839782 state 0x2 ref 0 00000100:00100000:1.0:1789760693.812830:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760693.813490:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129728 00000100:00100000:0.0:1789760693.813501:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129728 00000100:00100000:0.0:1789760693.813554:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 674988468722 time 675062839782 state 0x0 ref 1 00000100:00100000:0.0:1789760693.813570:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675063100198 time 675062839782 state 0x1 ref 1 00000100:00100000:0.0:1789760693.813609:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675063100198 time 675128911772 state 0x0 ref 1 00000100:00100000:0.0:1789760693.813620:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507129728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760693.813664:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16150528, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:0.0:1789760693.813672:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.813686:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760693.813690:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.813730:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.813745:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.868296:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16150528 -> 16154624 00010000:00010000:1.0:1789760693.868314:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868334:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868344:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868351:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.868357:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868367:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868386:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.868393:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b680c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16150528->16154623] (req 16150528->16154623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.868564:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507129728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54940us (55739us total) trans 4294971263 rc 0/0 00000100:00100000:1.0:1789760693.868574:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3973 00000100:00100000:1.0:1789760693.868584:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675063100198 time 675128911772 state 0x2 ref 0 00000100:00100000:0.0:1789760693.872856:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760693.872947:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129856 00000100:00100000:1.0:1789760693.872957:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129856 00000100:00100000:1.0:1789760693.873011:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675063100198 time 675128911772 state 0x0 ref 1 00000100:00100000:1.0:1789760693.873027:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675129172188 time 675128911772 state 0x1 ref 1 00000100:00100000:1.0:1789760693.873175:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675129172188 time 675188476921 state 0x0 ref 1 00000100:00100000:1.0:1789760693.873195:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507129856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.873245:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16154624, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.873254:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6813 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.873270:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:1.0:1789760693.873274:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.873281:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6813 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16154624->16158719] (req 16154624->16158719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.873295:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6813 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16154624->16158719] (req 16154624->16158719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760693.916029:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971264 is committed 00000100:00100000:1.0:1789760693.923157:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507129984 00000100:00100000:1.0:1789760693.923169:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507129984 00000100:00100000:1.0:1789760693.923230:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675129172188 time 675188476921 state 0x0 ref 1 00000100:00100000:1.0:1789760693.923246:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675188737337 time 675188476921 state 0x1 ref 1 00000100:00100000:1.0:1789760693.923276:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675188737337 time 675238579686 state 0x0 ref 1 00000100:00100000:1.0:1789760693.923290:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507129984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760693.923336:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16158720, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x8007 00010000:00010000:1.0:1789760693.923345:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.923361:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:1.0:1789760693.923365:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.923371:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.923385:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.971520:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16158720 -> 16162816 00010000:00010000:1.0:1789760693.971540:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971553:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971562:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971569:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760693.971574:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971584:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971772:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.971780:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b681a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16158720->16162815] (req 16158720->16162815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.972015:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507129984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48721us (49948us total) trans 4294971265 rc 0/0 00000100:00100000:1.0:1789760693.972026:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3975 00000100:00100000:1.0:1789760693.972036:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675188737337 time 675238579686 state 0x2 ref 0 00000001:00080000:1.0:1789760693.972083:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971265 is committed 00000100:00100000:1.0:1789760693.977539:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760693.982830:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760693.982852:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538252416:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760693.982915:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760693.982981:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538252416 00000100:00100000:1.0:1789760693.982989:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538252416 00000100:00100000:1.0:1789760693.983030:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760693.983039:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538252416:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760693.983081:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760693.983127:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538252416:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760693.983135:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760693.983204:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538252416:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760694.033978:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.077812:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971267 is committed 00000100:00100000:1.0:1789760694.082044:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.140046:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971268 is committed 00000100:00100000:0.0:1789760694.141420:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507130496 00000100:00100000:0.0:1789760694.141430:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507130496 00000100:00100000:0.0:1789760694.141484:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675349846584 time 675397712445 state 0x0 ref 1 00000100:00100000:0.0:1789760694.141498:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675397972861 time 675397712445 state 0x1 ref 1 00000100:00100000:0.0:1789760694.141526:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675397972861 time 675456829377 state 0x0 ref 1 00000100:00100000:0.0:1789760694.141538:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507130496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760694.141595:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16175104, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760694.141605:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6836 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.141620:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760694.141624:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.141630:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6836 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16175104->16179199] (req 16175104->16179199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.141643:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6836 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16175104->16179199] (req 16175104->16179199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.223419:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760694.261939:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.295452:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971271 is committed 00000100:00100000:1.0:1789760694.299515:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.352522:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971272 is committed 00002000:00010000:1.0:1789760694.352871:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16187392 -> 16191488 00010000:00010000:1.0:1789760694.352888:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352902:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352912:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352920:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760694.352925:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352934:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352953:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.352962:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b684b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16187392->16191487] (req 16187392->16191487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.353091:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507130880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53049us (53581us total) trans 4294971272 rc 0/0 00000100:00100000:1.0:1789760694.353102:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3982 00000100:00100000:1.0:1789760694.353112:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675577753190 time 675615328168 state 0x2 ref 0 00000100:00100000:0.0:1789760694.357371:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131008 00000100:00100000:0.0:1789760694.357380:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131008 00000100:00100000:0.0:1789760694.357429:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675577753190 time 675615328168 state 0x0 ref 1 00000100:00100000:0.0:1789760694.357443:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675615588584 time 675615328168 state 0x1 ref 1 00000100:00100000:0.0:1789760694.357500:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675615588584 time 675672803637 state 0x0 ref 1 00000100:00100000:0.0:1789760694.357513:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507131008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760694.357559:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16191488, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760694.357568:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.357583:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760694.357669:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.357675:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.357689:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760694.397864:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16191488 -> 16195584 00010000:00010000:0.0:1789760694.397886:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397899:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397908:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397916:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760694.397921:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397930:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397950:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.397958:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6852 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16191488->16195583] (req 16191488->16195583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.398042:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507131008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40526us (40773us total) trans 4294971273 rc 0/0 00000100:00100000:0.0:1789760694.398056:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3983 00000100:00100000:0.0:1789760694.398066:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675615588584 time 675672803637 state 0x2 ref 0 00000001:00080000:0.0:1789760694.398103:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971273 is committed 00000100:00100000:1.0:1789760694.404136:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760694.441717:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16195584 -> 16199680 00010000:00010000:0.0:1789760694.441736:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441749:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441758:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441765:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760694.441770:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441779:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441809:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.441816:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6859 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16195584->16199679] (req 16195584->16199679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.441916:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507131136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37534us (37784us total) trans 4294971274 rc 0/0 00000100:00100000:0.0:1789760694.441926:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3984 00000100:00100000:0.0:1789760694.441935:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675673064053 time 675719670378 state 0x2 ref 0 00000001:00080000:0.0:1789760694.441972:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971274 is committed 00000100:00100000:1.0:1789760694.452110:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760694.513134:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16199680 -> 16203776 00010000:00010000:1.0:1789760694.513162:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513277:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513289:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513297:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760694.513301:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513311:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513331:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.513338:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6860 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16199680->16203775] (req 16199680->16203775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.513436:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507131264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61089us (61331us total) trans 4294971275 rc 0/0 00000100:00100000:1.0:1789760694.513446:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3985 00000100:00100000:1.0:1789760694.513457:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675719930794 time 675767635397 state 0x2 ref 0 00000100:00100000:1.0:1789760694.517823:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760694.517992:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131392 00000100:00100000:0.0:1789760694.518002:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131392 00000100:00100000:0.0:1789760694.518073:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675719930794 time 675767635397 state 0x0 ref 1 00000100:00100000:0.0:1789760694.518091:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675767895813 time 675767635397 state 0x1 ref 1 00000100:00100000:0.0:1789760694.518124:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675767895813 time 675833427528 state 0x0 ref 1 00000100:00100000:0.0:1789760694.518137:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507131392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760694.518207:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16203776, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760694.518217:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.518232:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760694.518236:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.518242:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.518256:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760694.561778:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16203776 -> 16207872 00010000:00010000:1.0:1789760694.561805:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561818:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561827:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561834:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760694.561838:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561846:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561869:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.561876:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6867 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16203776->16207871] (req 16203776->16207871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.561966:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507131392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43826us (44147us total) trans 4294971276 rc 0/0 00000100:00100000:1.0:1789760694.561975:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3986 00000100:00100000:1.0:1789760694.561984:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675767895813 time 675833427528 state 0x2 ref 0 00000100:00100000:1.0:1789760694.565117:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131520 00000100:00100000:1.0:1789760694.565127:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131520 00000100:00100000:1.0:1789760694.565199:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675767895813 time 675833427528 state 0x0 ref 1 00000100:00100000:1.0:1789760694.565214:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675833687944 time 675833427528 state 0x1 ref 1 00000100:00100000:1.0:1789760694.565242:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675833687944 time 675880545294 state 0x0 ref 1 00000100:00100000:1.0:1789760694.565255:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760694.565301:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16207872, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:1.0:1789760694.565310:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.565326:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:1.0:1789760694.565331:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.565338:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.565350:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760694.603631:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16207872 -> 16211968 00010000:00010000:0.0:1789760694.603652:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.603666:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.603675:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.603683:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760694.603688:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.604095:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.604117:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.604125:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b686e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16207872->16211967] (req 16207872->16211967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.604354:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39095us (39377us total) trans 4294971277 rc 0/0 00000100:00100000:0.0:1789760694.604369:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3987 00000100:00100000:0.0:1789760694.604379:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675833687944 time 675880545294 state 0x2 ref 0 00000100:00100000:1.0:1789760694.608324:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131648 00000100:00100000:1.0:1789760694.608335:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131648 00000100:00100000:1.0:1789760694.608391:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675833687944 time 675880545294 state 0x0 ref 1 00000100:00100000:1.0:1789760694.608407:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675880805710 time 675880545294 state 0x1 ref 1 00000100:00100000:1.0:1789760694.608440:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675880805710 time 675923742743 state 0x0 ref 1 00000100:00100000:1.0:1789760694.608451:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760694.608497:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16211968, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:1.0:1789760694.608505:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.608520:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:1.0:1789760694.608524:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.608530:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.608542:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760694.681049:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16211968 -> 16216064 00010000:00010000:1.0:1789760694.681076:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681092:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681102:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681110:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760694.681115:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681123:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681151:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.681160:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6875 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16211968->16216063] (req 16211968->16216063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.682368:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507131648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73910us (74185us total) trans 4294971278 rc 0/0 00000100:00100000:1.0:1789760694.682384:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3988 00000100:00100000:1.0:1789760694.682392:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675880805710 time 675923742743 state 0x2 ref 0 00000001:00080000:0.0:1789760694.685292:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971278 is committed 00000100:00100000:1.0:1789760694.691610:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760694.692035:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131776 00000100:00100000:0.0:1789760694.692044:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131776 00000100:00100000:0.0:1789760694.692106:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675880805710 time 675923742743 state 0x0 ref 1 00000100:00100000:0.0:1789760694.692120:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675924003159 time 675923742743 state 0x1 ref 1 00000100:00100000:0.0:1789760694.692158:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675924003159 time 676007461332 state 0x0 ref 1 00000100:00100000:0.0:1789760694.692179:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760694.692237:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16216064, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760694.692253:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.692269:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760694.692273:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.692280:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.692296:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.748472:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971279 is committed 00002000:00010000:0.0:1789760694.750183:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16216064 -> 16220160 00010000:00010000:0.0:1789760694.750201:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750213:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750221:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750228:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760694.750232:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750242:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750261:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.750268:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b687c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16216064->16220159] (req 16216064->16220159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.750356:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58182us (58751us total) trans 4294971279 rc 0/0 00000100:00100000:0.0:1789760694.750366:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3989 00000100:00100000:0.0:1789760694.750375:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675924003159 time 676007461332 state 0x2 ref 0 00000100:00100000:0.0:1789760694.755476:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760694.755550:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507131904 00000100:00100000:1.0:1789760694.755559:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507131904 00000100:00100000:1.0:1789760694.755619:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 675924003159 time 676007461332 state 0x0 ref 1 00000100:00100000:1.0:1789760694.755635:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676007721748 time 676007461332 state 0x1 ref 1 00000100:00100000:1.0:1789760694.755664:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676007721748 time 676070967210 state 0x0 ref 1 00000100:00100000:1.0:1789760694.755677:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507131904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760694.755755:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16220160, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:1.0:1789760694.755765:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b6883 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.755894:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760694.755900:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.755907:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b6883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16220160->16224255] (req 16220160->16224255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.755939:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b6883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16220160->16224255] (req 16220160->16224255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.818866:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.881405:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971281 is committed 00000100:00100000:1.0:1789760694.886666:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760694.887032:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507132160 00000100:00100000:0.0:1789760694.887041:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507132160 00000100:00100000:0.0:1789760694.887092:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676071227626 time 676134539080 state 0x0 ref 1 00000100:00100000:0.0:1789760694.887107:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676134799496 time 676134539080 state 0x1 ref 1 00000100:00100000:0.0:1789760694.887146:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676134799496 time 676202440558 state 0x0 ref 1 00000100:00100000:0.0:1789760694.887159:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b458a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760694.887215:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16228352, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760694.887225:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.887239:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760694.887243:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.887249:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.887263:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760694.925062:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16228352 -> 16232448 00010000:00010000:1.0:1789760694.925083:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925097:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925106:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925129:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760694.925133:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925154:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925181:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.925190:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6891 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16228352->16232447] (req 16228352->16232447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.925285:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b458a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507132160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38123us (38624us total) trans 4294971282 rc 0/0 00000100:00100000:1.0:1789760694.925295:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3992 00000100:00100000:1.0:1789760694.925305:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676134799496 time 676202440558 state 0x2 ref 0 00000001:00080000:0.0:1789760694.925458:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971282 is committed 00000100:00100000:1.0:1789760694.930262:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760694.975990:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971283 is committed 00000100:00100000:0.0:1789760694.981112:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760695.031166:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16236544 -> 16240640 00010000:00010000:0.0:1789760695.031195:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031210:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031220:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031227:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760695.031232:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031251:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031271:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.031279:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b689f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16236544->16240639] (req 16236544->16240639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.031376:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507132416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49895us (50269us total) trans 4294971284 rc 0/0 00000100:00100000:0.0:1789760695.031386:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3994 00000100:00100000:0.0:1789760695.031396:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676246083382 time 676296767932 state 0x2 ref 0 00000001:00080000:1.0:1789760695.034115:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971284 is committed 00000100:00100000:1.0:1789760695.036510:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760695.036593:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507132544 00000100:00100000:0.0:1789760695.036603:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507132544 00000100:00100000:0.0:1789760695.036672:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676246083382 time 676296767932 state 0x0 ref 1 00000100:00100000:0.0:1789760695.036736:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676297028348 time 676296767932 state 0x1 ref 1 00000100:00100000:0.0:1789760695.036757:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676297028348 time 676352061362 state 0x0 ref 1 00000100:00100000:0.0:1789760695.036770:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132544:12345-192.168.203.41@tcp:4:dd.0 00000100:00080000:1.0:1789760695.037005:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760695.037081:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16240640, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x8007 00010000:00010000:0.0:1789760695.037091:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037107:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760695.037112:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.037118:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037132:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.078908:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971285 is committed 00002000:00010000:1.0:1789760695.079075:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16240640 -> 16244736 00002000:00010000:1.0:1789760695.079079:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760693 -> 1789760694 00002000:00010000:1.0:1789760695.079082:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760693 -> 1789760694 00010000:00010000:1.0:1789760695.079094:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079107:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079117:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079124:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760695.079129:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079155:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079182:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.079192:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b68a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16240640->16244735] (req 16240640->16244735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.079354:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42580us (42846us total) trans 4294971285 rc 0/0 00000100:00100000:1.0:1789760695.079365:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3995 00000100:00100000:1.0:1789760695.079374:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676297028348 time 676352061362 state 0x2 ref 0 00000100:00100000:1.0:1789760695.084054:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507132672 00000100:00100000:1.0:1789760695.084064:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507132672 00000100:00100000:1.0:1789760695.084125:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676297028348 time 676352061362 state 0x0 ref 1 00000100:00100000:1.0:1789760695.084153:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676352321778 time 676352061362 state 0x1 ref 1 00000100:00100000:1.0:1789760695.084190:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676352321778 time 676399493369 state 0x0 ref 1 00000100:00100000:1.0:1789760695.084203:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.084473:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16244736, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.084483:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.084994:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760695.085000:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.085007:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.085023:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.132165:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16244736 -> 16248832 00010000:00010000:1.0:1789760695.132242:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132255:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132264:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132271:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760695.132277:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132286:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132307:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.132315:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b68ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16244736->16248831] (req 16244736->16248831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.132452:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48247us (48524us total) trans 4294971286 rc 0/0 00000100:00100000:1.0:1789760695.132462:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3996 00000100:00100000:1.0:1789760695.132472:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676352321778 time 676399493369 state 0x2 ref 0 00000100:00100000:0.0:1789760695.141076:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00010000:00010000:1.0:1789760695.195982:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760695.196192:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507132928 00000100:00100000:1.0:1789760695.196200:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507132928 00000100:00100000:1.0:1789760695.196248:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676399753785 time 676456706633 state 0x0 ref 1 00000100:00100000:1.0:1789760695.196263:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676456967049 time 676456706633 state 0x1 ref 1 00000100:00100000:1.0:1789760695.196292:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676456967049 time 676511595793 state 0x0 ref 1 00000100:00100000:1.0:1789760695.196304:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.196351:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16252928, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.196359:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.196375:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760695.196379:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.196385:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.196399:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.288239:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16252928 -> 16257024 00010000:00010000:1.0:1789760695.288260:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288275:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288285:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288294:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760695.288300:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288310:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288336:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.288344:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16252928->16257023] (req 16252928->16257023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.288485:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507132928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 92175us (92367us total) trans 4294971288 rc 0/0 00000100:00100000:1.0:1789760695.288498:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 3998 00000100:00100000:1.0:1789760695.288511:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676456967049 time 676511595793 state 0x2 ref 0 00000100:00100000:1.0:1789760695.355900:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507133184 00000100:00100000:1.0:1789760695.355915:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507133184 00000100:00100000:1.0:1789760695.355987:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676511856209 time 676608274219 state 0x0 ref 1 00000100:00100000:1.0:1789760695.356007:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676608534635 time 676608274219 state 0x1 ref 1 00000100:00100000:1.0:1789760695.356045:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676608534635 time 676671347953 state 0x0 ref 1 00000100:00100000:1.0:1789760695.356059:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507133184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.356122:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16261120, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.356150:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.356177:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760695.356184:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.356191:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16261120->16265215] (req 16261120->16265215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.356208:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b68c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16261120->16265215] (req 16261120->16265215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.418378:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971290 is committed 00000100:00100000:1.0:1789760695.440340:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507133312 00000100:00100000:1.0:1789760695.440352:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507133312 00000100:00100000:1.0:1789760695.440424:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676608534635 time 676671347953 state 0x0 ref 1 00000100:00100000:1.0:1789760695.440442:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676671608369 time 676671347953 state 0x1 ref 1 00000100:00100000:1.0:1789760695.440475:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676671608369 time 676755778428 state 0x0 ref 1 00000100:00100000:1.0:1789760695.440489:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507133312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.440562:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16265216, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.440573:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.440600:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760695.440606:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.440613:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.440630:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.531887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971291 is committed 00002000:00010000:1.0:1789760695.532176:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16265216 -> 16269312 00010000:00010000:1.0:1789760695.532197:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532220:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532231:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532239:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760695.532245:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532256:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532280:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.532288:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b68d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16265216->16269311] (req 16265216->16269311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.532407:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507133312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91913us (92230us total) trans 4294971291 rc 0/0 00000100:00100000:1.0:1789760695.532420:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4001 00000100:00100000:1.0:1789760695.532432:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676671608369 time 676755778428 state 0x2 ref 0 00000100:00100000:0.0:1789760695.537821:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760695.643277:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971292 is committed 00000100:00100000:1.0:1789760695.812374:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507133696 00000100:00100000:1.0:1789760695.812384:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507133696 00000100:00100000:1.0:1789760695.812440:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676853782734 time 676967217121 state 0x0 ref 1 00000100:00100000:1.0:1789760695.812457:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676967477537 time 676967217121 state 0x1 ref 1 00000100:00100000:1.0:1789760695.812492:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676967477537 time 677127794757 state 0x0 ref 1 00000100:00100000:1.0:1789760695.812504:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507133696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.812553:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16277504, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.812573:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.812591:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760695.812596:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.812603:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.812618:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.866243:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16277504 -> 16281600 00010000:00010000:1.0:1789760695.866259:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866272:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866291:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866299:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760695.866304:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866312:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866330:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.866337:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16277504->16281599] (req 16277504->16281599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.866433:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507133696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53926us (54145us total) trans 4294971294 rc 0/0 00000100:00100000:1.0:1789760695.866444:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4004 00000100:00100000:1.0:1789760695.866453:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 676967477537 time 677127794757 state 0x2 ref 0 00002000:00010000:0.0:1789760695.913010:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16281600 -> 16285696 00010000:00010000:0.0:1789760695.913025:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913038:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913047:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913054:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760695.913058:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913073:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913094:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.913103:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b68ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16281600->16285695] (req 16281600->16285695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.913209:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507133824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32204us (32545us total) trans 4294971295 rc 0/0 00000100:00100000:0.0:1789760695.913219:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4005 00000100:00100000:0.0:1789760695.913229:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677128055173 time 677196294358 state 0x2 ref 0 00000100:00100000:1.0:1789760695.951995:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507134080 00000100:00100000:1.0:1789760695.952004:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507134080 00000100:00100000:1.0:1789760695.952054:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677196554774 time 677231690546 state 0x0 ref 1 00000100:00100000:1.0:1789760695.952068:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677231950962 time 677231690546 state 0x1 ref 1 00000100:00100000:1.0:1789760695.952099:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677231950962 time 677267402164 state 0x0 ref 1 00000100:00100000:1.0:1789760695.952110:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507134080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760695.952251:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16289792, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:1.0:1789760695.952261:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.952275:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760695.952279:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.952285:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16289792->16293887] (req 16289792->16293887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.952299:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b68fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16289792->16293887] (req 16289792->16293887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.983486:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971297 is committed 00000100:00100000:0.0:1789760695.986956:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507134208 00000100:00100000:0.0:1789760695.986967:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507134208 00000100:00100000:0.0:1789760695.987021:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677231950962 time 677267402164 state 0x0 ref 1 00000100:00100000:0.0:1789760695.987036:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677267662580 time 677267402164 state 0x1 ref 1 00000100:00100000:0.0:1789760695.987067:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677267662580 time 677302370038 state 0x0 ref 1 00000100:00100000:0.0:1789760695.987078:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507134208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760695.987124:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16293888, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8007 00010000:00010000:0.0:1789760695.987144:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.987158:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760695.987170:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.987177:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.987190:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.085305:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16293888 -> 16297984 00002000:00010000:0.0:1789760696.085310:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760694 -> 1789760695 00002000:00010000:0.0:1789760696.085313:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760694 -> 1789760695 00010000:00010000:0.0:1789760696.085327:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085342:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085351:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085359:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.085364:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085373:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085393:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.085401:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6901 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16293888->16297983] (req 16293888->16297983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.085488:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507134208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 98407us (98600us total) trans 4294971298 rc 0/0 00000100:00100000:0.0:1789760696.085498:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4008 00000100:00100000:0.0:1789760696.085508:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677267662580 time 677302370038 state 0x2 ref 0 00000100:00100000:1.0:1789760696.138199:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507134464 00000100:00100000:1.0:1789760696.138209:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507134464 00000100:00100000:1.0:1789760696.138259:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677302630454 time 677404354620 state 0x0 ref 1 00000100:00100000:1.0:1789760696.138274:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677404615036 time 677404354620 state 0x1 ref 1 00000100:00100000:1.0:1789760696.138312:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677404615036 time 677453615046 state 0x0 ref 1 00000100:00100000:1.0:1789760696.138324:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507134464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760696.138371:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16302080, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:1.0:1789760696.138380:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b690f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.138395:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760696.138399:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.138406:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b690f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16302080->16306175] (req 16302080->16306175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.138420:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b690f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16302080->16306175] (req 16302080->16306175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.224558:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760696.232377:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971301 is committed 00000100:00100000:0.0:1789760696.239894:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507134720 00000100:00100000:0.0:1789760696.239904:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507134720 00000100:00100000:0.0:1789760696.239955:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677453875462 time 677518684759 state 0x0 ref 1 00000100:00100000:0.0:1789760696.239969:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677518945175 time 677518684759 state 0x1 ref 1 00000100:00100000:0.0:1789760696.240033:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677518945175 time 677555335866 state 0x0 ref 1 00000100:00100000:0.0:1789760696.240047:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507134720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.240096:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16310272, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.240104:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b691d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.240118:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760696.240122:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.240139:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b691d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16310272->16314367] (req 16310272->16314367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.240154:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b691d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16310272->16314367] (req 16310272->16314367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.307657:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971302 is committed 00000001:00080000:1.0:1789760696.430527:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971303 is committed 00000100:00100000:1.0:1789760696.435400:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760696.494887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971304 is committed 00000100:00100000:1.0:1789760696.499486:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760696.546349:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16322560 -> 16326656 00010000:00010000:0.0:1789760696.546369:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546381:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546391:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546409:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.546414:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546423:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546443:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.546451:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6932 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16322560->16326655] (req 16322560->16326655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.546544:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507135104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46675us (47064us total) trans 4294971305 rc 0/0 00000100:00100000:0.0:1789760696.546553:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4015 00000100:00100000:0.0:1789760696.546563:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677751321451 time 677815155834 state 0x2 ref 0 00000001:00080000:0.0:1789760696.547856:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971305 is committed 00000100:00100000:0.0:1789760696.550182:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507135232 00000100:00100000:0.0:1789760696.550191:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507135232 00000100:00100000:0.0:1789760696.550242:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677751321451 time 677815155834 state 0x0 ref 1 00000100:00100000:0.0:1789760696.550260:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677815416250 time 677815155834 state 0x1 ref 1 00000100:00100000:0.0:1789760696.550290:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677815416250 time 677865593533 state 0x0 ref 1 00000100:00100000:0.0:1789760696.550303:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507135232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.550351:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16326656, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.550360:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.550375:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760696.550379:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.550385:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.550411:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.614635:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16326656 -> 16330752 00010000:00010000:0.0:1789760696.614654:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614667:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614677:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614720:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.614726:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614736:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614756:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.614764:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6939 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16326656->16330751] (req 16326656->16330751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.614850:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507135232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64544us (64784us total) trans 4294971306 rc 0/0 00000100:00100000:0.0:1789760696.614860:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4016 00000100:00100000:0.0:1789760696.614868:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677815416250 time 677865593533 state 0x2 ref 0 00000100:00100000:0.0:1789760696.621730:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507135360 00000100:00100000:0.0:1789760696.621740:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507135360 00000100:00100000:0.0:1789760696.621794:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677815416250 time 677865593533 state 0x0 ref 1 00000100:00100000:0.0:1789760696.621810:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677865853949 time 677865593533 state 0x1 ref 1 00000100:00100000:0.0:1789760696.621832:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677865853949 time 677937135775 state 0x0 ref 1 00000100:00100000:0.0:1789760696.621849:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507135360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.621896:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16330752, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.621905:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.621921:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760696.621925:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.621932:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.621947:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.629948:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971306 is committed 00002000:00010000:1.0:1789760696.683180:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16330752 -> 16334848 00010000:00010000:1.0:1789760696.683200:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683214:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683224:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683237:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760696.683242:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683251:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683271:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.683279:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6940 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16330752->16334847] (req 16330752->16334847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.683378:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507135360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61526us (61777us total) trans 4294971307 rc 0/0 00000100:00100000:1.0:1789760696.683390:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4017 00000100:00100000:1.0:1789760696.683410:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677865853949 time 677937135775 state 0x2 ref 0 00000100:00100000:0.0:1789760696.687738:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507135488 00000100:00100000:0.0:1789760696.687748:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507135488 00000100:00100000:0.0:1789760696.687800:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677865853949 time 677937135775 state 0x0 ref 1 00000100:00100000:0.0:1789760696.687815:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677937396191 time 677937135775 state 0x1 ref 1 00000100:00100000:0.0:1789760696.687844:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677937396191 time 678003148110 state 0x0 ref 1 00000100:00100000:0.0:1789760696.687857:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507135488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.687902:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16334848, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.687911:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.687931:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760696.687936:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.687942:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.687957:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.743318:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16334848 -> 16338944 00010000:00010000:0.0:1789760696.743334:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743347:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743356:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743363:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.743368:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743377:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743458:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.743468:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6947 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16334848->16338943] (req 16334848->16338943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.743555:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507135488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55697us (55966us total) trans 4294971308 rc 0/0 00000100:00100000:0.0:1789760696.743565:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4018 00000100:00100000:0.0:1789760696.743635:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 677937396191 time 678003148110 state 0x2 ref 0 00000100:00100000:0.0:1789760696.751485:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760696.815183:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16338944 -> 16343040 00010000:00010000:0.0:1789760696.815198:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815209:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815218:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815225:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.815230:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815237:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815260:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.815268:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b694e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16338944->16343039] (req 16338944->16343039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.815354:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507135616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63474us (63876us total) trans 4294971309 rc 0/0 00000100:00100000:0.0:1789760696.815364:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4019 00000100:00100000:0.0:1789760696.815373:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678003408526 time 678067169280 state 0x2 ref 0 00000100:00100000:0.0:1789760696.819623:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507135744 00000100:00100000:0.0:1789760696.819633:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507135744 00000100:00100000:0.0:1789760696.819687:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678003408526 time 678067169280 state 0x0 ref 1 00000100:00100000:0.0:1789760696.819717:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678067429696 time 678067169280 state 0x1 ref 1 00000100:00100000:0.0:1789760696.819746:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678067429696 time 678135049095 state 0x0 ref 1 00000100:00100000:0.0:1789760696.819758:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507135744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.819804:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16343040, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.819813:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.819827:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760696.819832:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.819837:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.819850:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.866062:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16343040 -> 16347136 00010000:00010000:0.0:1789760696.866077:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866091:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866100:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866107:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.866112:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866129:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866148:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.866213:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6955 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16343040->16347135] (req 16343040->16347135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.866343:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507135744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46582us (47051us total) trans 4294971310 rc 0/0 00000100:00100000:0.0:1789760696.866353:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4020 00000100:00100000:0.0:1789760696.866363:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678067429696 time 678135049095 state 0x2 ref 0 00000100:00100000:0.0:1789760696.870513:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507135872 00000100:00100000:0.0:1789760696.870523:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507135872 00000100:00100000:0.0:1789760696.870869:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678067429696 time 678135049095 state 0x0 ref 1 00000100:00100000:0.0:1789760696.870885:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678135309511 time 678135049095 state 0x1 ref 1 00000100:00100000:0.0:1789760696.870955:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678135309511 time 678186257529 state 0x0 ref 1 00000100:00100000:0.0:1789760696.870968:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507135872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.871014:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16347136, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.871022:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.871037:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760696.871041:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.871047:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.871061:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.904226:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16347136 -> 16351232 00010000:00010000:0.0:1789760696.904241:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904254:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904264:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904271:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760696.904277:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904286:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904306:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.904315:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b695c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16347136->16351231] (req 16347136->16351231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.904509:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507135872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33538us (34352us total) trans 4294971311 rc 0/0 00000100:00100000:0.0:1789760696.904523:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4021 00000100:00100000:0.0:1789760696.904532:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678135309511 time 678186257529 state 0x2 ref 0 00000001:00080000:1.0:1789760696.940166:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971312 is committed 00000100:00100000:1.0:1789760696.943245:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507136128 00000100:00100000:1.0:1789760696.943255:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507136128 00000100:00100000:1.0:1789760696.943307:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678186517945 time 678225164184 state 0x0 ref 1 00000100:00100000:1.0:1789760696.943321:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678225424600 time 678225164184 state 0x1 ref 1 00000100:00100000:1.0:1789760696.943351:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678225424600 time 678258654389 state 0x0 ref 1 00000100:00100000:1.0:1789760696.943362:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507136128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760696.943406:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16355328, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:1.0:1789760696.943415:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b696a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.943430:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760696.943434:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.943440:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b696a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16355328->16359423] (req 16355328->16359423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.943452:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b696a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16355328->16359423] (req 16355328->16359423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.973150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971313 is committed 00000100:00100000:0.0:1789760696.977646:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507136256 00000100:00100000:0.0:1789760696.977656:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507136256 00000100:00100000:0.0:1789760696.977849:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678225424600 time 678258654389 state 0x0 ref 1 00000100:00100000:0.0:1789760696.977864:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678258914805 time 678258654389 state 0x1 ref 1 00000100:00100000:0.0:1789760696.977918:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678258914805 time 678293221200 state 0x0 ref 1 00000100:00100000:0.0:1789760696.977931:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507136256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760696.977980:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16359424, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8007 00010000:00010000:0.0:1789760696.977990:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.978005:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760696.978010:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.978016:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.978031:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.006283:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16359424 -> 16363520 00010000:00010000:0.0:1789760697.006297:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006309:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006318:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006326:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.006331:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006338:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006358:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.006366:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6971 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16359424->16363519] (req 16359424->16363519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.006458:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507136256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28524us (28973us total) trans 4294971314 rc 0/0 00000100:00100000:0.0:1789760697.006468:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4024 00000100:00100000:0.0:1789760697.006477:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678258914805 time 678293221200 state 0x2 ref 0 00000100:00100000:1.0:1789760697.046584:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.046629:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.046665:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.046685:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.046724:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.046742:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507136512 00000100:00100000:1.0:1789760697.046745:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507136512 00000100:00100000:1.0:1789760697.046771:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 450 00000100:00100000:1.0:1789760697.046780:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507136512:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760697.046920:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507136512:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 141us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.046927:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 450 00000100:00100000:1.0:1789760697.046953:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 801 00000100:00100000:1.0:1789760697.046959:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b458700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507137024:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760697.046980:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b458700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507137024:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 21us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.046985:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 801 00002000:00010000:0.0:1789760697.050544:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16363520 -> 16367616 00002000:00010000:0.0:1789760697.050551:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760695 -> 1789760696 00002000:00010000:0.0:1789760697.050554:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760695 -> 1789760696 00010000:00010000:0.0:1789760697.050566:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050588:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050596:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050603:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.050607:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050616:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050635:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.050643:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6978 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16363520->16367615] (req 16363520->16367615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.050754:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507136384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40659us (40935us total) trans 4294971315 rc 0/0 00000100:00100000:0.0:1789760697.050766:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4025 00000100:00100000:0.0:1789760697.050775:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678293481616 time 678325384266 state 0x2 ref 0 00000001:00080000:1.0:1789760697.099890:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971316 is committed 00000100:00100000:0.0:1789760697.106029:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507137280 00000100:00100000:0.0:1789760697.106038:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507137280 00000100:00100000:0.0:1789760697.106088:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678325644682 time 678369087640 state 0x0 ref 1 00000100:00100000:0.0:1789760697.106104:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678369348056 time 678369087640 state 0x1 ref 1 00000100:00100000:0.0:1789760697.106144:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678369348056 time 678421447249 state 0x0 ref 1 00000100:00100000:0.0:1789760697.106166:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507137280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760697.106212:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16371712, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8007 00010000:00010000:0.0:1789760697.106222:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.106238:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760697.106242:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.106249:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.106263:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.150431:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971317 is committed 00002000:00010000:0.0:1789760697.150913:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16371712 -> 16375808 00010000:00010000:0.0:1789760697.150929:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150941:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150955:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150962:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.150967:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150976:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150995:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.151002:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6986 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16371712->16375807] (req 16371712->16375807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.151138:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507137280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44977us (45248us total) trans 4294971317 rc 0/0 00000100:00100000:0.0:1789760697.151148:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4027 00000100:00100000:0.0:1789760697.151165:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678369348056 time 678421447249 state 0x2 ref 0 00000001:00080000:1.0:1789760697.225985:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971318 is committed 00000100:00100000:0.0:1789760697.229683:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507137536 00000100:00100000:0.0:1789760697.229728:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507137536 00000100:00100000:0.0:1789760697.229782:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678421707665 time 678469900019 state 0x0 ref 1 00000100:00100000:0.0:1789760697.229797:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678470160435 time 678469900019 state 0x1 ref 1 00000100:00100000:0.0:1789760697.229822:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678470160435 time 678545125088 state 0x0 ref 1 00000100:00100000:0.0:1789760697.229835:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507137536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760697.229881:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16379904, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8007 00010000:00010000:0.0:1789760697.229890:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.229905:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760697.229910:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.229916:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.229931:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.284464:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971319 is committed 00002000:00010000:0.0:1789760697.284579:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16379904 -> 16384000 00010000:00010000:0.0:1789760697.284593:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284604:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284617:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284624:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.284629:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284638:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284655:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.284663:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6994 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16379904->16383999] (req 16379904->16383999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.284775:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507137536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54937us (55219us total) trans 4294971319 rc 0/0 00000100:00100000:0.0:1789760697.284785:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4029 00000100:00100000:0.0:1789760697.284794:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678470160435 time 678545125088 state 0x2 ref 0 00000100:00100000:0.0:1789760697.290902:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:1.0:1789760697.337736:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971320 is committed 00002000:00010000:0.0:1789760697.337986:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16384000 -> 16388096 00010000:00010000:0.0:1789760697.338003:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338020:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338029:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338103:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.338109:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338129:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338150:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338270:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b699b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16384000->16388095] (req 16384000->16388095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.338403:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507137664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47256us (47505us total) trans 4294971320 rc 0/0 00000100:00100000:0.0:1789760697.338413:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4030 00000100:00100000:0.0:1789760697.338423:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678545385504 time 678606416279 state 0x2 ref 0 00000100:00100000:0.0:1789760697.342449:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760697.424721:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16388096 -> 16392192 00010000:00010000:1.0:1789760697.424738:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424751:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424762:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424770:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760697.424776:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424786:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424812:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.424820:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b69a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16388096->16392191] (req 16388096->16392191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.424919:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507137792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 81366us (82477us total) trans 4294971321 rc 0/0 00000100:00100000:1.0:1789760697.424930:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4031 00000100:00100000:1.0:1789760697.424940:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678606676695 time 678658835458 state 0x2 ref 0 00000100:00100000:0.0:1789760697.428113:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760697.535383:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16392192 -> 16396288 00010000:00010000:0.0:1789760697.535406:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535420:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535429:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535437:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760697.535442:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535452:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535473:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.535480:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b69a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16392192->16396287] (req 16392192->16396287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.536339:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507137920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 107970us (108228us total) trans 4294971322 rc 0/0 00000100:00100000:0.0:1789760697.536351:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4032 00000100:00100000:0.0:1789760697.536360:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678659095874 time 678743653479 state 0x2 ref 0 00000100:00100000:0.0:1789760697.549856:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760697.565985:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b45bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538252672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760697.566051:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.566139:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538252672 00000100:00100000:1.0:1789760697.566147:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538252672 00000100:00100000:1.0:1789760697.566208:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1789760697.566217:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538252672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760697.566249:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760697.566294:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538252672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.566301:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00080000:1.0:1789760697.566321:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760697.566352:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b45bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538252672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760697.568936:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538252800 00000100:00100000:1.0:1789760697.568943:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538252800 00000100:00100000:1.0:1789760697.568986:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:1.0:1789760697.568994:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538252800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569049:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538252800:12345-0@lo:400:kworker.0 Request processed in 56us (1807us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.569056:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:1.0:1789760697.569088:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253312 00000100:00100000:1.0:1789760697.569092:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253312 00000100:00100000:1.0:1789760697.569131:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1789760697.569137:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538253312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569179:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538253312:12345-0@lo:400:kworker.0 Request processed in 42us (1896us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.569185:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00080000:1.0:1789760697.569208:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760697.569223:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538253056:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569255:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569278:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105823800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538253568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569302:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569318:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105820e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538254080:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569337:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569355:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1789760697.569362:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105820700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538253184:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569383:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569397:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105822d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538253696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569417:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569435:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105821c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538254208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569454:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.569480:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253056 00000100:00100000:1.0:1789760697.569483:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253056 00000100:00100000:1.0:1789760697.569503:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789760697.569509:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538253056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.569543:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538253056:12345-0@lo:400:kworker.0 Request processed in 34us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.569549:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789760697.570190:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253824 00000100:00100000:1.0:1789760697.570197:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253824 00000100:00100000:1.0:1789760697.570236:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253696 00000100:00100000:1.0:1789760697.570240:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253696 00000100:00100000:1.0:1789760697.570259:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538254208 00000100:00100000:1.0:1789760697.570263:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538254208 00000100:00100000:1.0:1789760697.570279:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789760697.570285:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538253824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.570330:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538253824:12345-0@lo:400:kworker.0 Request processed in 45us (3012us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.570337:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789760697.570349:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:1.0:1789760697.570355:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538253696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.570380:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538253696:12345-0@lo:400:kworker.0 Request processed in 26us (962us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.570386:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:1.0:1789760697.570395:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789760697.570401:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538254208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.570426:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538254208:12345-0@lo:400:kworker.0 Request processed in 26us (971us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.570432:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789760697.570459:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253568 00000100:00100000:1.0:1789760697.570463:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253568 00000100:00100000:1.0:1789760697.570486:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538254080 00000100:00100000:1.0:1789760697.570490:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538254080 00000100:00100000:1.0:1789760697.570501:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538253184 00000100:00100000:1.0:1789760697.570504:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538253184 00000100:00100000:1.0:1789760697.570514:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 805 00000100:00100000:1.0:1789760697.570519:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538253568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.570549:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538253568:12345-0@lo:400:kworker.0 Request processed in 29us (1245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.570555:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 805 00000100:00100000:0.0:1789760697.570728:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b45bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538254336:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760697.570762:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760697.571030:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 806 00000100:00100000:0.0:1789760697.571038:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538254080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.571074:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538254080:12345-0@lo:400:kworker.0 Request processed in 37us (1735us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760697.571080:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 806 00000100:00100000:1.0:1789760697.571257:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 807 00000100:00100000:1.0:1789760697.571265:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538253184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.571297:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538253184:12345-0@lo:400:kworker.0 Request processed in 33us (1913us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.571303:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 807 00000100:00100000:1.0:1789760697.588572:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538254336 00000100:00100000:1.0:1789760697.588586:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538254336 00000100:00100000:1.0:1789760697.588651:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789760697.588661:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538254336:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760697.588717:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760697.588798:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538254336:12345-0@lo:13:osp-pre-1-1.0 Request processed in 140us (18034us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.588804:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00080000:1.0:1789760697.588828:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760697.588834:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760697.588837:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760697.588875:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105820700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538253184:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.588889:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105822d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538253696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.588900:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105821c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538254208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.588912:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538253056:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.588923:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105823800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538253568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.588933:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105820e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538254080:0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.592328:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b45bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538254336:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760697.646990:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507138176 00000100:00100000:1.0:1789760697.647000:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507138176 00000100:00100000:1.0:1789760697.647053:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678743913895 time 678868845568 state 0x0 ref 1 00000100:00100000:1.0:1789760697.647068:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678869105984 time 678868845568 state 0x1 ref 1 00000100:00100000:1.0:1789760697.647098:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 678869105984 time 678962401287 state 0x0 ref 1 00000100:00100000:1.0:1789760697.647110:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507138176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760697.647167:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16400384, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8007 00010000:00010000:1.0:1789760697.647176:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b69b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.647191:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760697.647196:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.647202:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b69b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16400384->16404479] (req 16400384->16404479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.647220:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b69b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16400384->16404479] (req 16400384->16404479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.820532:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760697.934894:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760698.069349:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971327 is committed 00000100:00100000:0.0:1789760698.173656:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760698.243354:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16420864 -> 16424960 00010000:00010000:1.0:1789760698.243369:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243381:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243389:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243396:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760698.243401:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243409:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243440:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.243449:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b69da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16420864->16424959] (req 16420864->16424959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.243544:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507138816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69579us (69892us total) trans 4294971329 rc 0/0 00000100:00100000:1.0:1789760698.243553:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4039 00000100:00100000:1.0:1789760698.243572:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679404623179 time 679489251691 state 0x2 ref 0 00000100:00100000:0.0:1789760698.249550:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760698.298921:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16424960 -> 16429056 00010000:00010000:1.0:1789760698.298941:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.298954:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.298964:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.298973:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760698.298977:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.298987:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.299008:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.299022:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b69e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16424960->16429055] (req 16424960->16429055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.299133:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507138944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49287us (49588us total) trans 4294971330 rc 0/0 00000100:00100000:1.0:1789760698.299144:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4040 00000100:00100000:1.0:1789760698.299164:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679489512107 time 679565135265 state 0x2 ref 0 00000001:00080000:0.0:1789760698.350636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971331 is committed 00000100:00100000:0.0:1789760698.355269:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760698.355816:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507139200 00000100:00100000:1.0:1789760698.355826:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507139200 00000100:00100000:1.0:1789760698.355880:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679565395681 time 679619284025 state 0x0 ref 1 00000100:00100000:1.0:1789760698.355896:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679619544441 time 679619284025 state 0x1 ref 1 00000100:00100000:1.0:1789760698.355967:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679619544441 time 679671269428 state 0x0 ref 1 00000100:00100000:1.0:1789760698.355981:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507139200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760698.356040:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16433152, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:1.0:1789760698.356048:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.356071:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760698.356076:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.356083:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.356104:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.414772:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16433152 -> 16437248 00010000:00010000:1.0:1789760698.414793:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414807:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414818:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414826:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760698.414832:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414842:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414861:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.414870:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16433152->16437247] (req 16433152->16437247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.414969:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507139200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58985us (59704us total) trans 4294971332 rc 0/0 00000100:00100000:1.0:1789760698.414980:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4042 00000100:00100000:1.0:1789760698.414990:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679619544441 time 679671269428 state 0x2 ref 0 00000001:00080000:0.0:1789760698.415240:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971332 is committed 00000100:00100000:0.0:1789760698.422484:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760698.422675:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507139328 00000100:00100000:1.0:1789760698.422687:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507139328 00000100:00100000:1.0:1789760698.422767:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679619544441 time 679671269428 state 0x0 ref 1 00000100:00100000:1.0:1789760698.422782:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679671529844 time 679671269428 state 0x1 ref 1 00000100:00100000:1.0:1789760698.422804:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679671529844 time 679738107740 state 0x0 ref 1 00000100:00100000:1.0:1789760698.422816:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507139328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760698.422866:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16437248, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:1.0:1789760698.422874:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.422889:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760698.422893:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.422899:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16437248->16441343] (req 16437248->16441343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.422912:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b69f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16437248->16441343] (req 16437248->16441343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.497315:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971333 is committed 00000100:00100000:0.0:1789760698.502039:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507139456 00000100:00100000:0.0:1789760698.502049:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507139456 00000100:00100000:0.0:1789760698.502102:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679671529844 time 679738107740 state 0x0 ref 1 00000100:00100000:0.0:1789760698.502126:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679738368156 time 679738107740 state 0x1 ref 1 00000100:00100000:0.0:1789760698.502171:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679738368156 time 679817474836 state 0x0 ref 1 00000100:00100000:0.0:1789760698.502183:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507139456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760698.502227:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16441344, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:0.0:1789760698.502235:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.502251:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760698.502255:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.502261:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.502274:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.588283:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538254464:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760698.588351:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760698.589904:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105822a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538254592:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760698.589949:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:0.0:1789760698.597678:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971334 is committed 00002000:00010000:0.0:1789760698.597770:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16441344 -> 16445440 00010000:00010000:0.0:1789760698.597785:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597798:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597813:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597821:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760698.597827:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597836:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597857:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.597866:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b69fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16441344->16445439] (req 16441344->16445439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.597947:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507139456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 95761us (96012us total) trans 4294971334 rc 0/0 00000100:00100000:0.0:1789760698.597958:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4044 00000100:00100000:0.0:1789760698.597968:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679738368156 time 679817474836 state 0x2 ref 0 00000100:00100000:0.0:1789760698.603639:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538254464:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760698.603683:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105822a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538254592:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760698.603931:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507139584 00000100:00100000:1.0:1789760698.603937:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507139584 00000100:00100000:1.0:1789760698.603976:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679738368156 time 679817474836 state 0x0 ref 1 00000100:00100000:1.0:1789760698.603989:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679817735252 time 679817474836 state 0x1 ref 1 00000100:00100000:1.0:1789760698.604043:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679817735252 time 679919345571 state 0x0 ref 1 00000100:00100000:1.0:1789760698.604055:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507139584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760698.604106:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16445440, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:1.0:1789760698.604125:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6a04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.604140:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760698.604158:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.604165:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16445440->16449535] (req 16445440->16449535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.604179:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16445440->16449535] (req 16445440->16449535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.649320:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760698.718799:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507139840 00000100:00100000:0.0:1789760698.718809:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507139840 00000100:00100000:0.0:1789760698.718866:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679919605987 time 679964873619 state 0x0 ref 1 00000100:00100000:0.0:1789760698.718882:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679965134035 time 679964873619 state 0x1 ref 1 00000100:00100000:0.0:1789760698.718905:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679965134035 time 680034209016 state 0x0 ref 1 00000100:00100000:0.0:1789760698.718918:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507139840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760698.718967:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16453632, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:0.0:1789760698.718976:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.718991:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760698.718996:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.719002:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.719016:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.756947:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16453632 -> 16457728 00010000:00010000:0.0:1789760698.756962:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.756973:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.756986:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.756993:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760698.756997:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.757005:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.757023:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.757032:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16453632->16457727] (req 16453632->16457727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.757138:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507139840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38217us (38716us total) trans 4294971337 rc 0/0 00000100:00100000:0.0:1789760698.757158:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4047 00000100:00100000:0.0:1789760698.757169:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 679965134035 time 680034209016 state 0x2 ref 0 00000100:00100000:0.0:1789760698.761874:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:0.0:1789760698.833864:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16457728 -> 16461824 00010000:00010000:0.0:1789760698.833879:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833899:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833910:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833918:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760698.833923:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833934:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833954:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.833963:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6a19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16457728->16461823] (req 16457728->16461823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.834255:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507139968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72064us (72277us total) trans 4294971338 rc 0/0 00000100:00100000:0.0:1789760698.834267:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4048 00000100:00100000:0.0:1789760698.834277:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680034469432 time 680077361053 state 0x2 ref 0 00000100:00100000:0.0:1789760698.840766:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760698.906367:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971339 is committed 00000100:00100000:0.0:1789760698.910448:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760698.910525:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140224 00000100:00100000:1.0:1789760698.910535:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140224 00000100:00100000:1.0:1789760698.910597:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680077621469 time 680156691584 state 0x0 ref 1 00000100:00100000:1.0:1789760698.910612:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680156952000 time 680156691584 state 0x1 ref 1 00000100:00100000:1.0:1789760698.910641:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680156952000 time 680225944256 state 0x0 ref 1 00000100:00100000:1.0:1789760698.910654:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507140224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760698.910724:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16465920, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:1.0:1789760698.910733:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.910749:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760698.910754:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.910761:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.910775:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.963523:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16465920 -> 16470016 00010000:00010000:1.0:1789760698.963539:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963619:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963630:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963638:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760698.963642:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963651:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963672:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.963681:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6a27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16465920->16470015] (req 16465920->16470015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.963791:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507140224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53134us (53347us total) trans 4294971340 rc 0/0 00000100:00100000:1.0:1789760698.963801:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4050 00000100:00100000:1.0:1789760698.963813:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680156952000 time 680225944256 state 0x2 ref 0 00000100:00100000:0.0:1789760698.968465:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140352 00000100:00100000:0.0:1789760698.968476:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140352 00000100:00100000:0.0:1789760698.968532:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680156952000 time 680225944256 state 0x0 ref 1 00000100:00100000:0.0:1789760698.968547:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680226204672 time 680225944256 state 0x1 ref 1 00000100:00100000:0.0:1789760698.968587:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680226204672 time 680283889852 state 0x0 ref 1 00000100:00100000:0.0:1789760698.968600:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507140352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760698.968645:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16470016, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:0.0:1789760698.968654:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.968670:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760698.968675:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.968682:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.968728:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.037376:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16470016 -> 16474112 00010000:00010000:0.0:1789760699.037391:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037474:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037484:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037491:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760699.037495:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037503:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037523:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.037530:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6a2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16470016->16474111] (req 16470016->16474111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.037726:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507140352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69089us (69558us total) trans 4294971341 rc 0/0 00000100:00100000:0.0:1789760699.037737:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4051 00000100:00100000:0.0:1789760699.037746:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680226204672 time 680283889852 state 0x2 ref 0 00000100:00100000:1.0:1789760699.043089:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140480 00000100:00100000:1.0:1789760699.043100:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140480 00000100:00100000:1.0:1789760699.043289:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680226204672 time 680283889852 state 0x0 ref 1 00000100:00100000:1.0:1789760699.043306:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680284150268 time 680283889852 state 0x1 ref 1 00000100:00100000:1.0:1789760699.043341:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680284150268 time 680358643048 state 0x0 ref 1 00000100:00100000:1.0:1789760699.043358:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507140480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760699.043505:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16474112, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8007 00010000:00010000:1.0:1789760699.043514:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6a35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.043529:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760699.043533:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.043539:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16474112->16478207] (req 16474112->16478207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.043614:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b6a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16474112->16478207] (req 16474112->16478207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.114481:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760699.114500:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10802c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538254720:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760699.114571:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760699.114628:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538254720 00000100:00100000:0.0:1789760699.114633:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538254720 00000100:00100000:0.0:1789760699.114670:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789760699.114677:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538254720:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760699.114723:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760699.114761:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538254720:12345-0@lo:41:osp-pre-1-0.0 Request processed in 84us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760699.114767:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789760699.114805:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10802c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538254720:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760699.121352:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00010000:00010000:2.0:1789760699.208416:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6a3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16478208->16482303] (req 16478208->16482303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.208655:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507140608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 86980us (87307us total) trans 4294971343 rc 0/0 00000100:00100000:2.0:1789760699.208666:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4053 00000100:00100000:2.0:1789760699.208675:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680358903464 time 680436958824 state 0x2 ref 0 00000100:00100000:1.0:1789760699.212091:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140736 00000100:00100000:1.0:1789760699.212101:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140736 00000100:00100000:1.0:1789760699.212171:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680358903464 time 680436958824 state 0x0 ref 1 00000100:00100000:1.0:1789760699.212185:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680437219240 time 680436958824 state 0x1 ref 1 00000100:00100000:1.0:1789760699.212212:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680437219240 time 680527516202 state 0x0 ref 1 00000100:00100000:1.0:1789760699.212224:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507140736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760699.212270:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16482304, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:1.0:1789760699.212278:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.212293:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760699.212296:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.212302:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.212315:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.266515:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971344 is committed 00002000:00010000:1.0:1789760699.266684:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16482304 -> 16486400 00010000:00010000:1.0:1789760699.266716:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266730:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266739:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266746:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760699.266751:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266759:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266777:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.266785:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16482304->16486399] (req 16482304->16486399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.266874:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507140736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54646us (54916us total) trans 4294971344 rc 0/0 00000100:00100000:1.0:1789760699.266885:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4054 00000100:00100000:1.0:1789760699.266895:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680437219240 time 680527516202 state 0x2 ref 0 00000100:00100000:2.0:1789760699.275601:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140864 00000100:00100000:2.0:1789760699.275612:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140864 00000100:00100000:2.0:1789760699.275964:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680437219240 time 680527516202 state 0x0 ref 1 00000100:00100000:2.0:1789760699.275979:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680527776618 time 680527516202 state 0x1 ref 1 00000100:00100000:2.0:1789760699.276001:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680527776618 time 680591305512 state 0x0 ref 1 00000100:00100000:2.0:1789760699.276014:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507140864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.276062:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16486400, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.276070:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6a4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.276086:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760699.276090:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.276096:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16486400->16490495] (req 16486400->16490495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.276118:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16486400->16490495] (req 16486400->16490495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.302498:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760699.383682:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971345 is committed 00000100:00100000:2.0:1789760699.387465:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760699.387561:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507140992 00000100:00100000:1.0:1789760699.387570:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507140992 00000100:00100000:1.0:1789760699.387621:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680527776618 time 680591305512 state 0x0 ref 1 00000100:00100000:1.0:1789760699.387637:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680591565928 time 680591305512 state 0x1 ref 1 00000100:00100000:1.0:1789760699.387666:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680591565928 time 680702969527 state 0x0 ref 1 00000100:00100000:1.0:1789760699.387679:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507140992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760699.387763:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16490496, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:1.0:1789760699.387772:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.387787:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760699.387791:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.387797:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.387810:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.423098:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16490496 -> 16494592 00010000:00010000:2.0:1789760699.423125:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423148:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423156:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423163:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760699.423168:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423176:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423196:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.423202:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6a51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16490496->16494591] (req 16490496->16494591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.423293:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507140992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35612us (35834us total) trans 4294971346 rc 0/0 00000100:00100000:2.0:1789760699.423304:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4056 00000100:00100000:2.0:1789760699.423313:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680591565928 time 680702969527 state 0x2 ref 0 00000100:00100000:1.0:1789760699.428755:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760699.428895:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507141120 00000100:00100000:0.0:1789760699.428905:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507141120 00000100:00100000:0.0:1789760699.428959:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680591565928 time 680702969527 state 0x0 ref 1 00000100:00100000:0.0:1789760699.428974:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680703229943 time 680702969527 state 0x1 ref 1 00000100:00100000:0.0:1789760699.429335:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680703229943 time 680744635922 state 0x0 ref 1 00000100:00100000:0.0:1789760699.429350:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760699.429403:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16494592, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:0.0:1789760699.429412:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.429439:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760699.429445:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.429451:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.429466:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.466928:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16494592 -> 16498688 00010000:00010000:2.0:1789760699.466943:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.466956:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.466965:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.466973:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760699.466978:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.466988:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.467251:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.467262:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6a58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16494592->16498687] (req 16494592->16498687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.467385:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38034us (38635us total) trans 4294971347 rc 0/0 00000100:00100000:2.0:1789760699.467395:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4057 00000100:00100000:2.0:1789760699.467403:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680703229943 time 680744635922 state 0x2 ref 0 00000100:00100000:0.0:1789760699.470647:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:0.0:1789760699.525926:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971348 is committed 00000100:00100000:1.0:1789760699.534428:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760699.534574:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507141376 00000100:00100000:2.0:1789760699.534585:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507141376 00000100:00100000:2.0:1789760699.534639:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680744896338 time 680786163539 state 0x0 ref 1 00000100:00100000:2.0:1789760699.534654:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680786423955 time 680786163539 state 0x1 ref 1 00000100:00100000:2.0:1789760699.534683:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680786423955 time 680849986484 state 0x0 ref 1 00000100:00100000:2.0:1789760699.534726:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507141376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.534772:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16502784, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.534782:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6a66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.534798:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760699.534802:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.534808:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6a66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16502784->16506879] (req 16502784->16506879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.534823:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6a66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16502784->16506879] (req 16502784->16506879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.571004:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971349 is committed 00000100:00100000:1.0:1789760699.576003:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760699.578903:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507141504 00000100:00100000:2.0:1789760699.578917:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507141504 00000100:00100000:2.0:1789760699.578980:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680786423955 time 680849986484 state 0x0 ref 1 00000100:00100000:2.0:1789760699.578995:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680850246900 time 680849986484 state 0x1 ref 1 00000100:00100000:2.0:1789760699.579307:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680850246900 time 680894608456 state 0x0 ref 1 00000100:00100000:2.0:1789760699.579322:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507141504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.579367:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16506880, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.579375:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.579389:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760699.579393:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.579399:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.579411:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.611903:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760699.611926:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538254848:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760699.611990:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760699.612422:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538254848:0@lo:41:osp-pre-0-1.0 00002000:00010000:2.0:1789760699.617471:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16506880 -> 16510976 00010000:00010000:2.0:1789760699.617487:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617500:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617509:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617517:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760699.617521:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617530:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617551:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.617559:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16506880->16510975] (req 16506880->16510975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.617891:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507141504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38567us (41892us total) trans 4294971350 rc 0/0 00000100:00100000:2.0:1789760699.617901:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4060 00000100:00100000:2.0:1789760699.617910:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680850246900 time 680894608456 state 0x2 ref 0 00000100:00100000:0.0:1789760699.622954:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:2.0:1789760699.666875:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971351 is committed 00010000:00010000:3.0:1789760699.667132:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16510976->16515071] (req 16510976->16515071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.667149:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16510976->16515071] (req 16510976->16515071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.667237:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507141632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43549us (44287us total) trans 4294971351 rc 0/0 00000100:00100000:3.0:1789760699.667247:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4061 00000100:00100000:3.0:1789760699.667256:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680894868872 time 680938972651 state 0x2 ref 0 00000100:00100000:3.0:1789760699.671845:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760699.671925:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507141760 00000100:00100000:2.0:1789760699.671935:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507141760 00000100:00100000:2.0:1789760699.671989:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680894868872 time 680938972651 state 0x0 ref 1 00000100:00100000:2.0:1789760699.672004:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680939233067 time 680938972651 state 0x1 ref 1 00000100:00100000:2.0:1789760699.672044:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680939233067 time 680987347142 state 0x0 ref 1 00000100:00100000:2.0:1789760699.672056:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.672112:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16515072, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.672121:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.672144:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760699.672149:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.672155:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.672169:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.748816:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971352 is committed 00002000:00010000:3.0:1789760699.748899:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16515072 -> 16519168 00010000:00010000:3.0:1789760699.748916:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748931:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748943:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748951:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760699.748955:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748964:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748990:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.748998:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6a7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16515072->16519167] (req 16515072->16519167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.749087:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77028us (77247us total) trans 4294971352 rc 0/0 00000100:00100000:3.0:1789760699.749097:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4062 00000100:00100000:3.0:1789760699.749116:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680939233067 time 680987347142 state 0x2 ref 0 00000100:00100000:1.0:1789760699.754005:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760699.754106:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507141888 00000100:00100000:0.0:1789760699.754116:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507141888 00000100:00100000:0.0:1789760699.754178:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680939233067 time 680987347142 state 0x0 ref 1 00000100:00100000:0.0:1789760699.754193:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680987607558 time 680987347142 state 0x1 ref 1 00000100:00100000:0.0:1789760699.754258:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680987607558 time 681069561190 state 0x0 ref 1 00000100:00100000:0.0:1789760699.754270:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760699.754315:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16519168, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:0.0:1789760699.754323:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.754337:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760699.754341:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.754346:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.754360:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.794106:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971353 is committed 00002000:00010000:0.0:1789760699.794254:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16519168 -> 16523264 00010000:00010000:0.0:1789760699.794270:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794282:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794290:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794297:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760699.794301:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794309:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794326:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.794333:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16519168->16523263] (req 16519168->16523263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.795267:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507141888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40993us (41265us total) trans 4294971353 rc 0/0 00000100:00100000:0.0:1789760699.795280:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4063 00000100:00100000:0.0:1789760699.795289:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680987607558 time 681069561190 state 0x2 ref 0 00000100:00100000:1.0:1789760699.798312:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760699.800307:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142016 00000100:00100000:3.0:1789760699.800323:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142016 00000100:00100000:2.0:1789760699.800489:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 680987607558 time 681069561190 state 0x0 ref 1 00000100:00100000:2.0:1789760699.800515:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681069821606 time 681069561190 state 0x1 ref 1 00000100:00100000:2.0:1789760699.800579:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681069821606 time 681115881091 state 0x0 ref 1 00000100:00100000:2.0:1789760699.800593:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.800788:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16523264, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.800798:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.800812:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760699.800816:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.800822:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.800834:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760699.844384:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16523264 -> 16527360 00010000:00010000:3.0:1789760699.844403:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844416:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844426:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844433:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760699.844438:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844522:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844542:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.844620:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6a89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16523264->16527359] (req 16523264->16527359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.844785:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507142016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44190us (46476us total) trans 4294971354 rc 0/0 00000100:00100000:3.0:1789760699.844795:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4064 00000100:00100000:3.0:1789760699.844805:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681069821606 time 681115881091 state 0x2 ref 0 00000001:00080000:1.0:1789760699.844883:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971354 is committed 00000100:00100000:2.0:1789760699.848897:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760699.849096:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142144 00000100:00100000:3.0:1789760699.849115:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142144 00000100:00100000:3.0:1789760699.849236:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681069821606 time 681115881091 state 0x0 ref 1 00000100:00100000:3.0:1789760699.849252:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681116141507 time 681115881091 state 0x1 ref 1 00000100:00100000:3.0:1789760699.849324:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681116141507 time 681164626797 state 0x0 ref 1 00000100:00100000:3.0:1789760699.849336:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507142144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760699.849384:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16527360, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:3.0:1789760699.849393:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.849411:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760699.849415:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.849423:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.849437:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.892219:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971355 is committed 00002000:00010000:2.0:1789760699.892315:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16527360 -> 16531456 00010000:00010000:2.0:1789760699.892330:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892344:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892354:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892363:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760699.892368:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892377:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892398:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.892405:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16527360->16531455] (req 16527360->16531455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.892641:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507142144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43301us (43749us total) trans 4294971355 rc 0/0 00000100:00100000:2.0:1789760699.892652:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4065 00000100:00100000:2.0:1789760699.892662:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681116141507 time 681164626797 state 0x2 ref 0 00000100:00100000:2.0:1789760699.896179:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760699.896322:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142272 00000100:00100000:0.0:1789760699.896332:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142272 00000100:00100000:0.0:1789760699.896386:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681116141507 time 681164626797 state 0x0 ref 1 00000100:00100000:0.0:1789760699.896403:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681164887213 time 681164626797 state 0x1 ref 1 00000100:00100000:0.0:1789760699.896534:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681164887213 time 681211836323 state 0x0 ref 1 00000100:00100000:0.0:1789760699.896618:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507142272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760699.896670:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16531456, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:0.0:1789760699.896679:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.896724:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760699.896730:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.896737:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.896752:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.929903:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971356 is committed 00002000:00010000:2.0:1789760699.930007:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16531456 -> 16535552 00010000:00010000:2.0:1789760699.930023:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930036:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930050:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930057:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760699.930061:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930071:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930090:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.930107:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6a97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16531456->16535551] (req 16531456->16535551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.930207:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507142272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33657us (34032us total) trans 4294971356 rc 0/0 00000100:00100000:2.0:1789760699.930216:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4066 00000100:00100000:2.0:1789760699.930225:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681164887213 time 681211836323 state 0x2 ref 0 00000100:00100000:1.0:1789760699.934052:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760699.934171:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142400 00000100:00100000:3.0:1789760699.934181:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142400 00000100:00100000:3.0:1789760699.934235:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681164887213 time 681211836323 state 0x0 ref 1 00000100:00100000:3.0:1789760699.934251:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681212096739 time 681211836323 state 0x1 ref 1 00000100:00100000:3.0:1789760699.934282:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681212096739 time 681249584470 state 0x0 ref 1 00000100:00100000:3.0:1789760699.934295:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760699.934341:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16535552, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:3.0:1789760699.934350:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.934366:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760699.934371:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.934377:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.934392:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760699.969020:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16535552 -> 16539648 00010000:00010000:3.0:1789760699.969039:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969051:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969061:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969068:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760699.969073:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969081:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969109:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.969117:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6a9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16535552->16539647] (req 16535552->16539647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.969219:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507142400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34922us (35172us total) trans 4294971357 rc 0/0 00000100:00100000:3.0:1789760699.969229:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4067 00000100:00100000:3.0:1789760699.969239:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681212096739 time 681249584470 state 0x2 ref 0 00000001:00080000:2.0:1789760699.970998:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971357 is committed 00000100:00100000:1.0:1789760699.973502:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760699.973736:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142528 00000100:00100000:2.0:1789760699.973745:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142528 00000100:00100000:2.0:1789760699.973789:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681212096739 time 681249584470 state 0x0 ref 1 00000100:00100000:2.0:1789760699.973802:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681249844886 time 681249584470 state 0x1 ref 1 00000100:00100000:2.0:1789760699.973820:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681249844886 time 681289124778 state 0x0 ref 1 00000100:00100000:2.0:1789760699.973828:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760699.973867:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16539648, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:2.0:1789760699.973874:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.974008:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760699.974013:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.974020:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.974035:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760700.001155:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971358 is committed 00002000:00010000:2.0:1789760700.001286:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16539648 -> 16543744 00010000:00010000:2.0:1789760700.001302:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001316:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001326:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001337:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760700.001343:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001352:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001371:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001379:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b6aa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16539648->16543743] (req 16539648->16543743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.001505:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27671us (28009us total) trans 4294971358 rc 0/0 00000100:00100000:2.0:1789760700.001516:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4068 00000100:00100000:2.0:1789760700.001526:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681249844886 time 681289124778 state 0x2 ref 0 00000100:00100000:1.0:1789760700.007054:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760700.007144:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142656 00000100:00100000:0.0:1789760700.007154:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142656 00000100:00100000:0.0:1789760700.007209:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681249844886 time 681289124778 state 0x0 ref 1 00000100:00100000:0.0:1789760700.007223:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681289385194 time 681289124778 state 0x1 ref 1 00000100:00100000:0.0:1789760700.007251:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681289385194 time 681322554143 state 0x0 ref 1 00000100:00100000:0.0:1789760700.007261:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507142656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760700.007311:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16543744, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8007 00010000:00010000:0.0:1789760700.007321:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.007336:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760700.007341:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.007347:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.007361:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760700.046266:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971359 is committed 00002000:00010000:1.0:1789760700.046512:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16543744 -> 16547840 00002000:00010000:1.0:1789760700.046517:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760698 -> 1789760699 00002000:00010000:1.0:1789760700.046520:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760698 -> 1789760699 00010000:00010000:1.0:1789760700.046532:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046554:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046564:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046571:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.046576:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046586:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046605:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.046612:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6aac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16543744->16547839] (req 16543744->16547839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.046768:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507142656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39503us (39719us total) trans 4294971359 rc 0/0 00000100:00100000:1.0:1789760700.046778:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4069 00000100:00100000:1.0:1789760700.046787:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681289385194 time 681322554143 state 0x2 ref 0 00000100:00100000:1.0:1789760700.053223:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760700.053432:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142784 00000100:00100000:3.0:1789760700.053443:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142784 00000100:00100000:3.0:1789760700.053501:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681289385194 time 681322554143 state 0x0 ref 1 00000100:00100000:3.0:1789760700.053516:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681322814559 time 681322554143 state 0x1 ref 1 00000100:00100000:3.0:1789760700.053558:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681322814559 time 681368861055 state 0x0 ref 1 00000100:00100000:3.0:1789760700.053571:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760700.053620:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16547840, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:3.0:1789760700.053630:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.053645:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760700.053649:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.053656:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.053675:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.141344:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971360 is committed 00002000:00010000:3.0:1789760700.141502:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16547840 -> 16551936 00010000:00010000:3.0:1789760700.141520:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141533:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141543:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141551:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760700.141556:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141566:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141587:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.141596:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ab3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16547840->16551935] (req 16547840->16551935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760700.141876:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 88118us (88472us total) trans 4294971360 rc 0/0 00000100:00100000:3.0:1789760700.141889:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4070 00000100:00100000:3.0:1789760700.141899:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681322814559 time 681368861055 state 0x2 ref 0 00000100:00100000:2.0:1789760700.145221:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760700.145426:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507142912 00000100:00100000:3.0:1789760700.145435:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507142912 00000100:00100000:3.0:1789760700.145490:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681322814559 time 681368861055 state 0x0 ref 1 00000100:00100000:3.0:1789760700.145510:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681369121471 time 681368861055 state 0x1 ref 1 00000100:00100000:3.0:1789760700.145541:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681369121471 time 681460844457 state 0x0 ref 1 00000100:00100000:3.0:1789760700.145553:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760700.145600:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16551936, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:3.0:1789760700.145608:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.145623:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760700.145627:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.145633:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.145646:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.182873:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971361 is committed 00002000:00010000:3.0:1789760700.182873:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16551936 -> 16556032 00010000:00010000:3.0:1789760700.182889:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182902:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182912:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182919:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760700.182924:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182937:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182958:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.182966:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6aba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16551936->16556031] (req 16551936->16556031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760700.183060:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507142912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37505us (37844us total) trans 4294971361 rc 0/0 00000100:00100000:3.0:1789760700.183072:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4071 00000100:00100000:3.0:1789760700.183081:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681369121471 time 681460844457 state 0x2 ref 0 00000100:00100000:1.0:1789760700.188406:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760700.188498:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143040 00000100:00100000:0.0:1789760700.188509:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143040 00000100:00100000:0.0:1789760700.188576:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681369121471 time 681460844457 state 0x0 ref 1 00000100:00100000:0.0:1789760700.188591:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681461104873 time 681460844457 state 0x1 ref 1 00000100:00100000:0.0:1789760700.188797:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681461104873 time 681504098733 state 0x0 ref 1 00000100:00100000:0.0:1789760700.189018:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507143040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760700.189070:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16556032, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:0.0:1789760700.189079:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.189108:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760700.189114:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.189121:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.189136:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.220803:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16556032 -> 16560128 00010000:00010000:1.0:1789760700.220823:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220836:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220851:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220859:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.220864:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220873:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220891:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.220899:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6ac1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16556032->16560127] (req 16556032->16560127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.221024:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507143040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32210us (32622us total) trans 4294971362 rc 0/0 00000100:00100000:1.0:1789760700.221034:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4072 00000100:00100000:1.0:1789760700.221044:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681461104873 time 681504098733 state 0x2 ref 0 00000001:00080000:3.0:1789760700.221128:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971362 is committed 00000100:00100000:1.0:1789760700.226534:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760700.226646:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143168 00000100:00100000:3.0:1789760700.226656:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143168 00000100:00100000:3.0:1789760700.226775:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681461104873 time 681504098733 state 0x0 ref 1 00000100:00100000:3.0:1789760700.226789:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681504359149 time 681504098733 state 0x1 ref 1 00000100:00100000:1.0:1789760700.226910:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681504359149 time 681542152011 state 0x0 ref 1 00000100:00100000:1.0:1789760700.226925:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240b8fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507143168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760700.226974:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16560128, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:1.0:1789760700.226983:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.226999:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760700.227004:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.227009:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.227021:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760700.264425:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971363 is committed 00002000:00010000:1.0:1789760700.264538:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16560128 -> 16564224 00010000:00010000:1.0:1789760700.264553:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264566:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264577:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264584:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.264589:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264598:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264618:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.264624:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ac8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16560128->16564223] (req 16560128->16564223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.264763:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240b8fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507143168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37837us (38232us total) trans 4294971363 rc 0/0 00000100:00100000:1.0:1789760700.264773:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4073 00000100:00100000:1.0:1789760700.264793:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681504359149 time 681542152011 state 0x2 ref 0 00000100:00100000:2.0:1789760700.269588:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760700.269676:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143296 00000100:00100000:3.0:1789760700.269686:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143296 00000100:00100000:3.0:1789760700.269766:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681504359149 time 681542152011 state 0x0 ref 1 00000100:00100000:3.0:1789760700.269792:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681542412427 time 681542152011 state 0x1 ref 1 00000100:00100000:3.0:1789760700.269822:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681542412427 time 681585125790 state 0x0 ref 1 00000100:00100000:3.0:1789760700.269836:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760700.269884:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16564224, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:3.0:1789760700.269904:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.269920:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760700.269924:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.269930:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.269946:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.322285:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760700.339541:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16564224 -> 16568320 00010000:00010000:1.0:1789760700.339562:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339576:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339585:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339593:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.339598:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339612:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339631:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.339640:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6acf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16564224->16568319] (req 16564224->16568319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.339788:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507143296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69940us (70195us total) trans 4294971364 rc 0/0 00000100:00100000:1.0:1789760700.339800:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4074 00000100:00100000:1.0:1789760700.339810:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681542412427 time 681585125790 state 0x2 ref 0 00000001:00080000:3.0:1789760700.339960:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971364 is committed 00000100:00100000:2.0:1789760700.343319:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760700.345298:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143424 00000100:00100000:1.0:1789760700.345313:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143424 00000100:00100000:1.0:1789760700.345379:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681542412427 time 681585125790 state 0x0 ref 1 00000100:00100000:1.0:1789760700.345394:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681585386206 time 681585125790 state 0x1 ref 1 00000100:00100000:1.0:1789760700.345432:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681585386206 time 681660734575 state 0x0 ref 1 00000100:00100000:1.0:1789760700.345447:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760700.345494:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16568320, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:1.0:1789760700.345503:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.345521:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760700.345525:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.345532:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.345547:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.447614:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16568320 -> 16572416 00010000:00010000:1.0:1789760700.447635:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447651:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447660:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447666:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.447670:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447680:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447718:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.447727:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6ad6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16568320->16572415] (req 16568320->16572415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760700.447734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971365 is committed 00000100:00100000:1.0:1789760700.449305:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 103852us (105986us total) trans 4294971365 rc 0/0 00000100:00100000:1.0:1789760700.449319:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4075 00000100:00100000:1.0:1789760700.449331:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681585386206 time 681660734575 state 0x2 ref 0 00000100:00100000:2.0:1789760700.456309:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760700.457962:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143552 00000100:00100000:1.0:1789760700.457974:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143552 00000100:00100000:1.0:1789760700.458040:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681585386206 time 681660734575 state 0x0 ref 1 00000100:00100000:1.0:1789760700.458057:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681660994991 time 681660734575 state 0x1 ref 1 00000100:00100000:1.0:1789760700.458087:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681660994991 time 681773390135 state 0x0 ref 1 00000100:00100000:1.0:1789760700.458109:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760700.458166:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16572416, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:1.0:1789760700.458176:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.458192:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760700.458196:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.458202:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.458220:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760700.518755:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971366 is committed 00002000:00010000:1.0:1789760700.519291:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16572416 -> 16576512 00010000:00010000:1.0:1789760700.519307:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519320:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519330:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519337:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.519342:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519350:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519369:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.519377:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6add lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16572416->16576511] (req 16572416->16576511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.519469:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61359us (63167us total) trans 4294971366 rc 0/0 00000100:00100000:1.0:1789760700.519479:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4076 00000100:00100000:1.0:1789760700.519489:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681660994991 time 681773390135 state 0x2 ref 0 00000100:00100000:2.0:1789760700.528422:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760700.528507:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143680 00000100:00100000:1.0:1789760700.528517:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143680 00000100:00100000:1.0:1789760700.528572:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681660994991 time 681773390135 state 0x0 ref 1 00000100:00100000:1.0:1789760700.528587:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681773650551 time 681773390135 state 0x1 ref 1 00000100:00100000:1.0:1789760700.528618:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681773650551 time 681843921090 state 0x0 ref 1 00000100:00100000:1.0:1789760700.528631:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507143680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760700.528678:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16576512, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:1.0:1789760700.528688:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.528732:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760700.528738:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.528744:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.528758:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.606411:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971367 is committed 00002000:00010000:1.0:1789760700.606610:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16576512 -> 16580608 00010000:00010000:1.0:1789760700.606625:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606638:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606648:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606655:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760700.606660:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606669:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606688:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.606729:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ae4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16576512->16580607] (req 16576512->16580607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.608002:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507143680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78259us (78475us total) trans 4294971367 rc 0/0 00000100:00100000:1.0:1789760700.608019:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4077 00000100:00100000:1.0:1789760700.608030:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681773650551 time 681843921090 state 0x2 ref 0 00000100:00100000:2.0:1789760700.610324:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760700.610756:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143808 00000100:00100000:1.0:1789760700.610766:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143808 00000100:00100000:1.0:1789760700.610902:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681773650551 time 681843921090 state 0x0 ref 1 00000100:00100000:1.0:1789760700.610920:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681844181506 time 681843921090 state 0x1 ref 1 00000100:00100000:2.0:1789760700.611239:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681844181506 time 681926539174 state 0x0 ref 1 00000100:00100000:2.0:1789760700.611254:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760700.611299:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16580608, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:2.0:1789760700.611307:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.611323:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760700.611328:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.611334:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.611347:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.719054:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16580608 -> 16584704 00010000:00010000:2.0:1789760700.719074:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719086:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719106:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719113:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760700.719117:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719126:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719212:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.719220:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6aeb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16580608->16584703] (req 16580608->16584703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.719316:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507143808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 108060us (109005us total) trans 4294971368 rc 0/0 00000100:00100000:2.0:1789760700.719325:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4078 00000100:00100000:2.0:1789760700.719334:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681844181506 time 681926539174 state 0x2 ref 0 00000001:00080000:0.0:1789760700.719419:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971368 is committed 00000100:00100000:2.0:1789760700.732253:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760700.732320:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507143936 00000100:00100000:1.0:1789760700.732331:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507143936 00000100:00100000:1.0:1789760700.732384:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681844181506 time 681926539174 state 0x0 ref 1 00000100:00100000:1.0:1789760700.732400:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681926799590 time 681926539174 state 0x1 ref 1 00000100:00100000:1.0:1789760700.732469:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681926799590 time 682047771457 state 0x0 ref 1 00000100:00100000:1.0:1789760700.732482:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507143936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760700.732532:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16584704, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:1.0:1789760700.732552:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.732568:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760700.732573:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.732579:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.732600:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760700.809470:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16584704 -> 16588800 00010000:00010000:0.0:1789760700.809492:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809506:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809515:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809529:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760700.809543:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809553:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809573:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809581:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6af2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16584704->16588799] (req 16584704->16588799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.809689:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507143936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77203us (77440us total) trans 4294971369 rc 0/0 00000100:00100000:0.0:1789760700.809728:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4079 00000100:00100000:0.0:1789760700.809739:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681926799590 time 682047771457 state 0x2 ref 0 00000001:00080000:0.0:1789760700.809789:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971369 is committed 00000100:00100000:1.0:1789760700.813172:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760700.813311:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144064 00000100:00100000:2.0:1789760700.813322:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144064 00000100:00100000:2.0:1789760700.813379:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 681926799590 time 682047771457 state 0x0 ref 1 00000100:00100000:2.0:1789760700.813394:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682048031873 time 682047771457 state 0x1 ref 1 00000100:00100000:2.0:1789760700.813420:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682048031873 time 682128723256 state 0x0 ref 1 00000100:00100000:2.0:1789760700.813433:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760700.813483:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16588800, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:2.0:1789760700.813492:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.813508:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760700.813512:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.813519:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.813534:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.873568:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971370 is committed 00002000:00010000:0.0:1789760700.873672:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16588800 -> 16592896 00010000:00010000:0.0:1789760700.873688:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873720:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873735:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873742:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760700.873747:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873757:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873777:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.873785:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6af9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16588800->16592895] (req 16588800->16592895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.873882:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60447us (60715us total) trans 4294971370 rc 0/0 00000100:00100000:0.0:1789760700.873891:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4080 00000100:00100000:0.0:1789760700.873902:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682048031873 time 682128723256 state 0x2 ref 0 00000100:00100000:2.0:1789760700.879621:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760700.879734:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144192 00000100:00100000:0.0:1789760700.879743:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144192 00000100:00100000:0.0:1789760700.879796:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682048031873 time 682128723256 state 0x0 ref 1 00000100:00100000:0.0:1789760700.879811:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682128983672 time 682128723256 state 0x1 ref 1 00000100:00100000:0.0:1789760700.879846:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682128983672 time 682195148454 state 0x0 ref 1 00000100:00100000:0.0:1789760700.879858:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760700.879907:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16592896, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:0.0:1789760700.879916:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.879931:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760700.879935:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.879942:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.879955:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.973446:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971371 is committed 00002000:00010000:0.0:1789760700.973561:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16592896 -> 16596992 00010000:00010000:0.0:1789760700.973577:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973591:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973600:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973608:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760700.973614:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973623:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973644:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.973653:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16592896->16596991] (req 16592896->16596991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.973781:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 93919us (94164us total) trans 4294971371 rc 0/0 00000100:00100000:0.0:1789760700.973791:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4081 00000100:00100000:0.0:1789760700.973801:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682128983672 time 682195148454 state 0x2 ref 0 00000100:00100000:1.0:1789760700.987899:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760700.988121:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144320 00000100:00100000:0.0:1789760700.988131:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144320 00000100:00100000:0.0:1789760700.988188:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682128983672 time 682195148454 state 0x0 ref 1 00000100:00100000:0.0:1789760700.988203:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682195408870 time 682195148454 state 0x1 ref 1 00000100:00100000:0.0:1789760700.988238:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682195408870 time 682303540698 state 0x0 ref 1 00000100:00100000:0.0:1789760700.988252:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507144320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760700.988306:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16596992, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:0.0:1789760700.988316:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.988332:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760700.988336:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.988342:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.988358:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.046111:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971372 is committed 00002000:00010000:1.0:1789760701.046269:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16596992 -> 16601088 00010000:00010000:1.0:1789760701.046285:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046297:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046306:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046313:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760701.046318:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046327:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046346:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.046353:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6b07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16596992->16601087] (req 16596992->16601087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.046438:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507144320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58186us (58547us total) trans 4294971372 rc 0/0 00000100:00100000:1.0:1789760701.046448:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4082 00000100:00100000:1.0:1789760701.046458:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682195408870 time 682303540698 state 0x2 ref 0 00000100:00100000:3.0:1789760701.051919:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760701.052235:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144448 00000100:00100000:0.0:1789760701.052246:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144448 00000100:00100000:0.0:1789760701.052345:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682195408870 time 682303540698 state 0x0 ref 1 00000100:00100000:0.0:1789760701.052360:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682303801114 time 682303540698 state 0x1 ref 1 00000100:00100000:0.0:1789760701.052389:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682303801114 time 682367692971 state 0x0 ref 1 00000100:00100000:0.0:1789760701.052402:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507144448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760701.052450:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16601088, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8007 00010000:00010000:0.0:1789760701.052459:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.052475:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760701.052480:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.052486:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.052500:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760701.061544:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760701.096953:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971373 is committed 00002000:00010000:0.0:1789760701.097062:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16601088 -> 16605184 00002000:00010000:0.0:1789760701.097067:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760699 -> 1789760700 00002000:00010000:0.0:1789760701.097069:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760699 -> 1789760700 00010000:00010000:0.0:1789760701.097081:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097103:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097113:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097121:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760701.097135:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097145:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097165:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.097179:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6b0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16601088->16605183] (req 16601088->16605183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.097277:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507144448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44871us (45362us total) trans 4294971373 rc 0/0 00000100:00100000:0.0:1789760701.097287:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4083 00000100:00100000:0.0:1789760701.097296:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682303801114 time 682367692971 state 0x2 ref 0 00000100:00100000:3.0:1789760701.104202:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760701.105209:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144576 00000100:00100000:0.0:1789760701.105219:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144576 00000100:00100000:0.0:1789760701.105274:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682303801114 time 682367692971 state 0x0 ref 1 00000100:00100000:0.0:1789760701.105290:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682367953387 time 682367692971 state 0x1 ref 1 00000100:00100000:0.0:1789760701.105310:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682367953387 time 682420614273 state 0x0 ref 1 00000100:00100000:0.0:1789760701.105321:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760701.105369:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16605184, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:0.0:1789760701.105378:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.105393:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760701.105397:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.105403:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.105421:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.153304:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971374 is committed 00002000:00010000:0.0:1789760701.153479:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16605184 -> 16609280 00010000:00010000:0.0:1789760701.153495:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153508:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153517:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153524:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760701.153529:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153549:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153571:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.153579:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6b15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16605184->16609279] (req 16605184->16609279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.153679:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48353us (49481us total) trans 4294971374 rc 0/0 00000100:00100000:0.0:1789760701.153689:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4084 00000100:00100000:0.0:1789760701.153742:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682367953387 time 682420614273 state 0x2 ref 0 00000100:00100000:0.0:1789760701.159034:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.159172:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144704 00000100:00100000:2.0:1789760701.159182:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144704 00000100:00100000:2.0:1789760701.159241:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682367953387 time 682420614273 state 0x0 ref 1 00000100:00100000:2.0:1789760701.159258:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682420874689 time 682420614273 state 0x1 ref 1 00000100:00100000:2.0:1789760701.159292:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682420874689 time 682474594640 state 0x0 ref 1 00000100:00100000:2.0:1789760701.159306:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507144704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.159359:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16609280, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.159370:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.159387:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.159522:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.159532:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.159617:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760701.210970:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16609280 -> 16613376 00010000:00010000:1.0:1789760701.211002:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211016:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211025:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211032:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760701.211037:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211047:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211071:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.211078:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6b1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16609280->16613375] (req 16609280->16613375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.211303:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507144704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51995us (52274us total) trans 4294971375 rc 0/0 00000100:00100000:1.0:1789760701.211313:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4085 00000100:00100000:1.0:1789760701.211322:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682420874689 time 682474594640 state 0x2 ref 0 00000001:00080000:3.0:1789760701.215234:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971375 is committed 00000100:00100000:1.0:1789760701.215403:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.215525:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144832 00000100:00100000:2.0:1789760701.215544:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144832 00000100:00100000:2.0:1789760701.215595:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682420874689 time 682474594640 state 0x0 ref 1 00000100:00100000:2.0:1789760701.215609:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682474855056 time 682474594640 state 0x1 ref 1 00000100:00100000:2.0:1789760701.215638:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682474855056 time 682530941680 state 0x0 ref 1 00000100:00100000:2.0:1789760701.215652:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.215730:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16613376, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.215740:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.215754:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.215758:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.215764:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.215777:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.247724:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971376 is committed 00002000:00010000:2.0:1789760701.247841:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16613376 -> 16617472 00010000:00010000:2.0:1789760701.247856:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247867:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247875:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247882:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.247886:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247894:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247913:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.247920:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16613376->16617471] (req 16613376->16617471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.248145:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507144832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32490us (32747us total) trans 4294971376 rc 0/0 00000100:00100000:2.0:1789760701.248158:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4086 00000100:00100000:2.0:1789760701.248168:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682474855056 time 682530941680 state 0x2 ref 0 00000100:00100000:0.0:1789760701.252687:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.252884:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507144960 00000100:00100000:2.0:1789760701.252895:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507144960 00000100:00100000:2.0:1789760701.252950:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682474855056 time 682530941680 state 0x0 ref 1 00000100:00100000:2.0:1789760701.252965:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682531202096 time 682530941680 state 0x1 ref 1 00000100:00100000:2.0:1789760701.253113:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682531202096 time 682568414454 state 0x0 ref 1 00000100:00100000:2.0:1789760701.253127:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507144960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.253184:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16617472, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.253193:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.253208:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.253213:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.253219:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.253232:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760701.302031:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16617472 -> 16621568 00010000:00010000:2.0:1789760701.302051:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302064:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302075:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302082:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.302097:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302107:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302136:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.302145:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16617472->16621567] (req 16617472->16621567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.302241:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507144960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49111us (49558us total) trans 4294971377 rc 0/0 00000100:00100000:2.0:1789760701.302252:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4087 00000100:00100000:2.0:1789760701.302261:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682531202096 time 682568414454 state 0x2 ref 0 00000001:00080000:2.0:1789760701.304077:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971377 is committed 00000100:00100000:0.0:1789760701.318976:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760701.319145:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145088 00000100:00100000:1.0:1789760701.319155:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145088 00000100:00100000:1.0:1789760701.319208:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682531202096 time 682568414454 state 0x0 ref 1 00000100:00100000:1.0:1789760701.319224:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682568674870 time 682568414454 state 0x1 ref 1 00000100:00100000:1.0:1789760701.319255:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682568674870 time 682634558782 state 0x0 ref 1 00000100:00100000:1.0:1789760701.319268:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507145088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760701.319315:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16621568, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:1.0:1789760701.319324:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.319344:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760701.319348:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.319354:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.319367:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.341723:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760701.360206:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971378 is committed 00002000:00010000:1.0:1789760701.360339:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16621568 -> 16625664 00010000:00010000:1.0:1789760701.360354:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360366:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360374:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360381:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760701.360386:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360394:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360413:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.360420:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16621568->16625663] (req 16621568->16625663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.360510:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507145088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41240us (41540us total) trans 4294971378 rc 0/0 00000100:00100000:1.0:1789760701.360520:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4088 00000100:00100000:1.0:1789760701.360529:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682568674870 time 682634558782 state 0x2 ref 0 00000100:00100000:3.0:1789760701.364282:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.364737:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145216 00000100:00100000:2.0:1789760701.364748:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145216 00000100:00100000:2.0:1789760701.364803:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682568674870 time 682634558782 state 0x0 ref 1 00000100:00100000:2.0:1789760701.364818:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682634819198 time 682634558782 state 0x1 ref 1 00000100:00100000:2.0:1789760701.364851:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682634819198 time 682680153147 state 0x0 ref 1 00000100:00100000:2.0:1789760701.364865:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507145216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.365604:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16625664, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.365614:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.365832:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.365836:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.365843:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.365859:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.438535:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971379 is committed 00002000:00010000:2.0:1789760701.438722:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16625664 -> 16629760 00010000:00010000:2.0:1789760701.438737:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438750:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438760:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438767:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.438772:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438781:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438800:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.438808:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16625664->16629759] (req 16625664->16629759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.441251:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507145216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 74026us (74614us total) trans 4294971379 rc 0/0 00000100:00100000:2.0:1789760701.441267:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4089 00000100:00100000:2.0:1789760701.441278:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682634819198 time 682680153147 state 0x2 ref 0 00000100:00100000:0.0:1789760701.446420:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.446493:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145344 00000100:00100000:2.0:1789760701.446504:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145344 00000100:00100000:2.0:1789760701.446560:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682634819198 time 682680153147 state 0x0 ref 1 00000100:00100000:2.0:1789760701.446576:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682680413563 time 682680153147 state 0x1 ref 1 00000100:00100000:2.0:1789760701.446608:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682680413563 time 682761911275 state 0x0 ref 1 00000100:00100000:2.0:1789760701.446620:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.446670:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16629760, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.446678:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.446734:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.446740:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.446747:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.446762:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.498813:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971380 is committed 00002000:00010000:2.0:1789760701.498996:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16629760 -> 16633856 00010000:00010000:2.0:1789760701.499012:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499025:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499035:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499043:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.499049:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499057:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499076:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.499084:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6b3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16629760->16633855] (req 16629760->16633855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.499186:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52563us (52772us total) trans 4294971380 rc 0/0 00000100:00100000:2.0:1789760701.499196:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4090 00000100:00100000:2.0:1789760701.499205:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682680413563 time 682761911275 state 0x2 ref 0 00000100:00100000:0.0:1789760701.515825:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760701.515920:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145472 00000100:00100000:1.0:1789760701.515930:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145472 00000100:00100000:1.0:1789760701.515983:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682680413563 time 682761911275 state 0x0 ref 1 00000100:00100000:1.0:1789760701.515998:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682762171691 time 682761911275 state 0x1 ref 1 00000100:00100000:1.0:1789760701.516029:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682762171691 time 682831332159 state 0x0 ref 1 00000100:00100000:1.0:1789760701.516042:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760701.516097:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16633856, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:1.0:1789760701.516107:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.516123:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760701.516127:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.516133:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.516152:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.607127:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971381 is committed 00002000:00010000:1.0:1789760701.608749:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16633856 -> 16637952 00010000:00010000:1.0:1789760701.608767:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608780:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608790:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608797:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760701.608802:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608812:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608833:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.608840:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6b46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16633856->16637951] (req 16633856->16637951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.608941:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 92896us (93120us total) trans 4294971381 rc 0/0 00000100:00100000:1.0:1789760701.608951:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4091 00000100:00100000:1.0:1789760701.608961:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682762171691 time 682831332159 state 0x2 ref 0 00000100:00100000:3.0:1789760701.612985:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.613229:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145600 00000100:00100000:2.0:1789760701.613239:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145600 00000100:00100000:2.0:1789760701.613299:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682762171691 time 682831332159 state 0x0 ref 1 00000100:00100000:2.0:1789760701.613315:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682831592575 time 682831332159 state 0x1 ref 1 00000100:00100000:2.0:1789760701.613345:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682831592575 time 682928647723 state 0x0 ref 1 00000100:00100000:2.0:1789760701.613358:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.613408:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16637952, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.613417:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.613432:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.613436:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.613442:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.613458:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760701.665443:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16637952 -> 16642048 00010000:00010000:2.0:1789760701.665464:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665478:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665488:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665496:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.665501:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665509:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665547:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.665556:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16637952->16642047] (req 16637952->16642047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.665728:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971382 is committed 00000100:00100000:2.0:1789760701.666111:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507145600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52749us (53130us total) trans 4294971382 rc 0/0 00000100:00100000:2.0:1789760701.666130:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4092 00000100:00100000:2.0:1789760701.666140:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682831592575 time 682928647723 state 0x2 ref 0 00000100:00100000:2.0:1789760701.675459:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760701.675610:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145728 00000100:00100000:0.0:1789760701.675620:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145728 00000100:00100000:0.0:1789760701.675673:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682831592575 time 682928647723 state 0x0 ref 1 00000100:00100000:0.0:1789760701.675687:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682928908139 time 682928647723 state 0x1 ref 1 00000100:00100000:2.0:1789760701.675914:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682928908139 time 682991215269 state 0x0 ref 1 00000100:00100000:2.0:1789760701.675930:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507145728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.675979:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16642048, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.675989:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.676004:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760701.676009:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.676015:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.676029:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.754988:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971383 is committed 00002000:00010000:2.0:1789760701.755105:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16642048 -> 16646144 00010000:00010000:2.0:1789760701.755120:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755133:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755143:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755150:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.755155:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755164:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755185:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.755193:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6b54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16642048->16646143] (req 16642048->16646143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.755288:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507145728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79357us (79833us total) trans 4294971383 rc 0/0 00000100:00100000:2.0:1789760701.755300:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4093 00000100:00100000:2.0:1789760701.755309:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682928908139 time 682991215269 state 0x2 ref 0 00000100:00100000:0.0:1789760701.764421:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760701.764524:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145856 00000100:00100000:1.0:1789760701.764543:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145856 00000100:00100000:1.0:1789760701.764606:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682928908139 time 682991215269 state 0x0 ref 1 00000100:00100000:1.0:1789760701.764622:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682991475685 time 682991215269 state 0x1 ref 1 00000100:00100000:1.0:1789760701.764649:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682991475685 time 683079952429 state 0x0 ref 1 00000100:00100000:1.0:1789760701.764660:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507145856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760701.764739:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16646144, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:1.0:1789760701.764749:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.764764:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760701.764769:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.764775:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.764790:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.812855:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971384 is committed 00002000:00010000:0.0:1789760701.812966:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16646144 -> 16650240 00010000:00010000:0.0:1789760701.812982:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.812994:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.813009:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.813017:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760701.813022:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.813032:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.813052:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.813061:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6b5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16646144->16650239] (req 16646144->16650239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.813173:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507145856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48508us (48762us total) trans 4294971384 rc 0/0 00000100:00100000:0.0:1789760701.813183:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4094 00000100:00100000:0.0:1789760701.813193:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682991475685 time 683079952429 state 0x2 ref 0 00000100:00100000:1.0:1789760701.818404:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760701.820736:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507145984 00000100:00100000:2.0:1789760701.820750:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507145984 00000100:00100000:2.0:1789760701.820813:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 682991475685 time 683079952429 state 0x0 ref 1 00000100:00100000:2.0:1789760701.820829:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683080212845 time 683079952429 state 0x1 ref 1 00000100:00100000:2.0:1789760701.820873:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683080212845 time 683136175900 state 0x0 ref 1 00000100:00100000:2.0:1789760701.820887:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507145984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760701.820933:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16650240, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:2.0:1789760701.820941:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.820958:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760701.820962:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.820968:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.820983:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.881468:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971385 is committed 00002000:00010000:2.0:1789760701.881733:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16650240 -> 16654336 00010000:00010000:2.0:1789760701.881749:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881762:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881772:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881780:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760701.881785:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881793:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881814:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.881822:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6b62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16650240->16654335] (req 16650240->16654335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.881910:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507145984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61021us (63511us total) trans 4294971385 rc 0/0 00000100:00100000:2.0:1789760701.881921:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4095 00000100:00100000:2.0:1789760701.881930:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683080212845 time 683136175900 state 0x2 ref 0 00000100:00100000:2.0:1789760701.894168:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760701.894292:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146112 00000100:00100000:0.0:1789760701.894302:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146112 00000100:00100000:0.0:1789760701.894352:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683080212845 time 683136175900 state 0x0 ref 1 00000100:00100000:0.0:1789760701.894365:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683136436316 time 683136175900 state 0x1 ref 1 00000100:00100000:0.0:1789760701.894394:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683136436316 time 683209697580 state 0x0 ref 1 00000100:00100000:0.0:1789760701.894405:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507146112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760701.894457:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16654336, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:0.0:1789760701.894466:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.894480:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760701.894484:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.894490:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.894503:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.959516:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16654336 -> 16658432 00010000:00010000:0.0:1789760701.959544:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959558:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959568:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959575:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760701.959581:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959591:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959608:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.959616:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6b69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16654336->16658431] (req 16654336->16658431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.959728:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507146112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65319us (65564us total) trans 4294971386 rc 0/0 00000100:00100000:0.0:1789760701.959750:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4096 00000100:00100000:0.0:1789760701.959759:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683136436316 time 683209697580 state 0x2 ref 0 00000001:00080000:1.0:1789760701.963108:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971386 is committed 00000100:00100000:0.0:1789760701.963362:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760701.963446:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146240 00000100:00100000:1.0:1789760701.963456:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146240 00000100:00100000:1.0:1789760701.963507:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683136436316 time 683209697580 state 0x0 ref 1 00000100:00100000:1.0:1789760701.963521:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683209957996 time 683209697580 state 0x1 ref 1 00000100:00100000:1.0:1789760701.963591:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683209957996 time 683278893992 state 0x0 ref 1 00000100:00100000:1.0:1789760701.963603:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507146240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760701.963646:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16658432, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:1.0:1789760701.963654:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.963669:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760701.963673:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.963680:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.963826:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.988100:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971387 is committed 00002000:00010000:1.0:1789760701.988258:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16658432 -> 16662528 00010000:00010000:1.0:1789760701.988274:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988287:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988295:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988303:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760701.988308:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988318:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988336:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.988344:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16658432->16662527] (req 16658432->16662527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.988472:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507146240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24865us (25108us total) trans 4294971387 rc 0/0 00000100:00100000:1.0:1789760701.988484:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4097 00000100:00100000:1.0:1789760701.988493:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683209957996 time 683278893992 state 0x2 ref 0 00000100:00100000:2.0:1789760701.992493:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760701.992627:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146368 00000100:00100000:0.0:1789760701.992636:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146368 00000100:00100000:0.0:1789760701.992687:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683209957996 time 683278893992 state 0x0 ref 1 00000100:00100000:0.0:1789760701.992843:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683279154408 time 683278893992 state 0x1 ref 1 00000100:00100000:3.0:1789760701.992956:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683279154408 time 683308256822 state 0x0 ref 1 00000100:00100000:3.0:1789760701.992971:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507146368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760701.993022:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16662528, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8007 00010000:00010000:3.0:1789760701.993031:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.993046:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760701.993052:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.993058:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.993071:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760702.019356:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16662528 -> 16666624 00002000:00010000:3.0:1789760702.019432:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760700 -> 1789760701 00002000:00010000:3.0:1789760702.019435:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760700 -> 1789760701 00010000:00010000:3.0:1789760702.019450:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019464:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019473:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019480:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.019485:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019493:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019512:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.019520:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16662528->16666623] (req 16662528->16666623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.019680:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507146368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26706us (27190us total) trans 4294971388 rc 0/0 00000100:00100000:3.0:1789760702.019730:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4098 00000100:00100000:3.0:1789760702.019741:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683279154408 time 683308256822 state 0x2 ref 0 00000001:00080000:2.0:1789760702.022449:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971388 is committed 00000100:00100000:2.0:1789760702.022978:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760702.023073:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146496 00000100:00100000:0.0:1789760702.023092:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146496 00000100:00100000:0.0:1789760702.023213:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683279154408 time 683308256822 state 0x0 ref 1 00000100:00100000:0.0:1789760702.023230:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683308517238 time 683308256822 state 0x1 ref 1 00000100:00100000:0.0:1789760702.023291:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683308517238 time 683338594059 state 0x0 ref 1 00000100:00100000:0.0:1789760702.023303:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507146496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760702.023351:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16666624, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:0.0:1789760702.023360:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.023376:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760702.023380:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.023386:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.023400:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.054454:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971389 is committed 00002000:00010000:0.0:1789760702.054594:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16666624 -> 16670720 00010000:00010000:0.0:1789760702.054611:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054623:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054631:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054642:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760702.054646:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054655:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054674:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.054682:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6b7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16666624->16670719] (req 16666624->16670719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.054829:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507146496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31522us (31852us total) trans 4294971389 rc 0/0 00000100:00100000:0.0:1789760702.054840:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4099 00000100:00100000:0.0:1789760702.054851:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683308517238 time 683338594059 state 0x2 ref 0 00000100:00100000:0.0:1789760702.058973:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760702.059054:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146624 00000100:00100000:1.0:1789760702.059062:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146624 00000100:00100000:1.0:1789760702.059125:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683308517238 time 683338594059 state 0x0 ref 1 00000100:00100000:1.0:1789760702.059140:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683338854475 time 683338594059 state 0x1 ref 1 00000100:00100000:1.0:1789760702.059169:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683338854475 time 683374472527 state 0x0 ref 1 00000100:00100000:1.0:1789760702.059179:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507146624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760702.059225:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16670720, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:1.0:1789760702.059233:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.059248:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760702.059252:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.059258:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.059271:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.087506:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16670720 -> 16674816 00010000:00010000:1.0:1789760702.087524:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087552:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087561:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087569:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760702.087573:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087582:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087601:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.087609:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6b85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16670720->16674815] (req 16670720->16674815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.087728:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507146624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28545us (28760us total) trans 4294971390 rc 0/0 00000100:00100000:1.0:1789760702.087737:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4100 00000100:00100000:1.0:1789760702.087747:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683338854475 time 683374472527 state 0x2 ref 0 00000001:00080000:3.0:1789760702.090866:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971390 is committed 00000100:00100000:1.0:1789760702.091070:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.091185:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146752 00000100:00100000:3.0:1789760702.091193:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146752 00000100:00100000:3.0:1789760702.091240:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683338854475 time 683374472527 state 0x0 ref 1 00000100:00100000:3.0:1789760702.091255:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683374732943 time 683374472527 state 0x1 ref 1 00000100:00100000:3.0:1789760702.091283:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683374732943 time 683406587148 state 0x0 ref 1 00000100:00100000:3.0:1789760702.091294:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507146752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.091336:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16674816, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.091345:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.091360:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760702.091365:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.091370:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.091384:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760702.121294:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16674816 -> 16678912 00010000:00010000:3.0:1789760702.121313:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121327:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121336:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121344:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.121348:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121358:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121378:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.121386:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6b8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16674816->16678911] (req 16674816->16678911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.121485:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507146752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30187us (30416us total) trans 4294971391 rc 0/0 00000100:00100000:3.0:1789760702.121496:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4101 00000100:00100000:3.0:1789760702.121505:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683374732943 time 683406587148 state 0x2 ref 0 00000001:00080000:2.0:1789760702.121638:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971391 is committed 00000100:00100000:0.0:1789760702.125372:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.125465:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507146880 00000100:00100000:2.0:1789760702.125476:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507146880 00000100:00100000:2.0:1789760702.125528:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683374732943 time 683406587148 state 0x0 ref 1 00000100:00100000:2.0:1789760702.125556:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683406847564 time 683406587148 state 0x1 ref 1 00000100:00100000:2.0:1789760702.125588:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683406847564 time 683440891033 state 0x0 ref 1 00000100:00100000:2.0:1789760702.125600:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507146880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.125651:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16678912, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.125660:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.125676:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.125681:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.125688:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.125731:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.154725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971392 is committed 00002000:00010000:2.0:1789760702.154818:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16678912 -> 16683008 00010000:00010000:2.0:1789760702.154834:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154847:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154856:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154863:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760702.154868:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154878:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154896:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.154903:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6b93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16678912->16683007] (req 16678912->16683007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.154996:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507146880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29393us (29629us total) trans 4294971392 rc 0/0 00000100:00100000:2.0:1789760702.155010:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4102 00000100:00100000:2.0:1789760702.155019:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683406847564 time 683440891033 state 0x2 ref 0 00000100:00100000:0.0:1789760702.158104:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.158383:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147008 00000100:00100000:2.0:1789760702.158393:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147008 00000100:00100000:2.0:1789760702.158448:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683406847564 time 683440891033 state 0x0 ref 1 00000100:00100000:2.0:1789760702.158462:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683441151449 time 683440891033 state 0x1 ref 1 00000100:00100000:2.0:1789760702.158592:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683441151449 time 683473829480 state 0x0 ref 1 00000100:00100000:2.0:1789760702.158609:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507147008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.158657:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16683008, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.158665:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.158680:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.158685:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.158761:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.158778:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.168158:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.168198:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147264 00000100:00100000:2.0:1789760702.168209:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147264 00000100:00100000:2.0:1789760702.168277:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 808 00000100:00100000:2.0:1789760702.168289:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507147264:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760702.168364:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507147264:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 77us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760702.168373:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 808 00000100:00080000:2.0:1789760702.168395:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760702.171888:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760702.171927:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760702.171948:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760702.171972:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760702.171998:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147136 00000100:00100000:1.0:1789760702.172007:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147136 00000100:00100000:1.0:1789760702.172055:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 452 00000100:00100000:1.0:1789760702.172064:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507147136:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760702.172151:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507147136:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 88us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.172158:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 452 00000100:00100000:1.0:1789760702.172189:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147520 00000100:00100000:1.0:1789760702.172193:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147520 00000100:00100000:1.0:1789760702.172225:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147392 00000100:00100000:1.0:1789760702.172229:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147392 00000100:00100000:1.0:1789760702.172247:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147648 00000100:00100000:1.0:1789760702.172250:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147648 00000100:00100000:1.0:1789760702.172265:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 809 00000100:00100000:1.0:1789760702.172272:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507147520:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760702.172304:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507147520:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 33us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.172310:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 809 00000100:00100000:1.0:1789760702.172324:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 810 00000100:00100000:1.0:1789760702.172329:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507147392:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760702.172350:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507147392:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 21us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.172360:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 810 00000100:00100000:1.0:1789760702.172371:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 811 00000100:00100000:1.0:1789760702.172377:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507147648:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760702.172400:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507147648:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.172406:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 811 00000100:00080000:1.0:1789760702.172523:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760702.173407:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760702.195367:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971393 is committed 00002000:00010000:2.0:1789760702.195447:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16683008 -> 16687104 00010000:00010000:2.0:1789760702.195463:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195474:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195482:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195489:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760702.195494:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195503:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195524:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.195656:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6b9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16683008->16687103] (req 16683008->16687103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.195849:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507147008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37237us (37747us total) trans 4294971393 rc 0/0 00000100:00100000:2.0:1789760702.195859:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4103 00000100:00100000:2.0:1789760702.195869:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683441151449 time 683473829480 state 0x2 ref 0 00000100:00100000:1.0:1789760702.200003:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.200103:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147776 00000100:00100000:3.0:1789760702.200112:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147776 00000100:00100000:3.0:1789760702.200230:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683441151449 time 683473829480 state 0x0 ref 1 00000100:00100000:3.0:1789760702.200248:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683474089896 time 683473829480 state 0x1 ref 1 00000100:00100000:3.0:1789760702.200316:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683474089896 time 683515618270 state 0x0 ref 1 00000100:00100000:3.0:1789760702.200329:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507147776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.200376:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16687104, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.200385:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.200401:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760702.200406:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.200412:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.200426:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.234139:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971394 is committed 00002000:00010000:2.0:1789760702.234400:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16687104 -> 16691200 00010000:00010000:2.0:1789760702.234415:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234429:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234438:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234446:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760702.234450:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234458:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234482:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.234489:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ba1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16687104->16691199] (req 16687104->16691199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.234740:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507147776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34407us (34741us total) trans 4294971394 rc 0/0 00000100:00100000:2.0:1789760702.234751:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4104 00000100:00100000:2.0:1789760702.234760:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683474089896 time 683515618270 state 0x2 ref 0 00000100:00100000:0.0:1789760702.238978:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.239187:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507147904 00000100:00100000:2.0:1789760702.239197:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507147904 00000100:00100000:2.0:1789760702.239254:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683474089896 time 683515618270 state 0x0 ref 1 00000100:00100000:2.0:1789760702.239269:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683515878686 time 683515618270 state 0x1 ref 1 00000100:00100000:2.0:1789760702.239335:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683515878686 time 683554637902 state 0x0 ref 1 00000100:00100000:2.0:1789760702.239358:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507147904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.239408:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16691200, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.239417:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.239433:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.239438:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.239445:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.239458:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.294416:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971395 is committed 00002000:00010000:3.0:1789760702.294512:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16691200 -> 16695296 00010000:00010000:3.0:1789760702.294636:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294650:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294660:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294667:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.294671:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294679:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294730:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.294738:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ba8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16691200->16695295] (req 16691200->16695295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.294821:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507147904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55470us (55848us total) trans 4294971395 rc 0/0 00000100:00100000:3.0:1789760702.294835:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4105 00000100:00100000:3.0:1789760702.294846:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683515878686 time 683554637902 state 0x2 ref 0 00000100:00100000:1.0:1789760702.298493:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.298773:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148032 00000100:00100000:2.0:1789760702.298782:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148032 00000100:00100000:2.0:1789760702.298832:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683515878686 time 683554637902 state 0x0 ref 1 00000100:00100000:2.0:1789760702.298847:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683554898318 time 683554637902 state 0x1 ref 1 00000100:00100000:2.0:1789760702.298907:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683554898318 time 683614209336 state 0x0 ref 1 00000100:00100000:2.0:1789760702.298919:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.298967:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16695296, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.298976:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.299000:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.299005:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.299012:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.299026:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.346617:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971396 is committed 00002000:00010000:3.0:1789760702.346908:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16695296 -> 16699392 00010000:00010000:3.0:1789760702.346924:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.346937:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.346947:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.346955:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.346959:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.346968:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.346998:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.347007:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6baf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16695296->16699391] (req 16695296->16699391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.347127:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48197us (48630us total) trans 4294971396 rc 0/0 00000100:00100000:3.0:1789760702.347138:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4106 00000100:00100000:3.0:1789760702.347148:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683554898318 time 683614209336 state 0x2 ref 0 00000100:00100000:1.0:1789760702.351037:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760702.352789:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148160 00000100:00100000:2.0:1789760702.352802:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148160 00000100:00100000:2.0:1789760702.352866:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683554898318 time 683614209336 state 0x0 ref 1 00000100:00100000:2.0:1789760702.352882:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683614469752 time 683614209336 state 0x1 ref 1 00000100:00100000:2.0:1789760702.352915:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683614469752 time 683668218719 state 0x0 ref 1 00000100:00100000:2.0:1789760702.352929:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.352978:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16699392, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.352997:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.353013:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.353018:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.353025:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.353040:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.366434:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760702.414115:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971397 is committed 00002000:00010000:3.0:1789760702.414185:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16699392 -> 16703488 00010000:00010000:3.0:1789760702.414206:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414220:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414229:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414237:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.414241:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414250:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414270:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.414282:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6bb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16699392->16703487] (req 16699392->16703487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.414376:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61445us (63343us total) trans 4294971397 rc 0/0 00000100:00100000:3.0:1789760702.414386:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4107 00000100:00100000:3.0:1789760702.414396:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683614469752 time 683668218719 state 0x2 ref 0 00000100:00100000:0.0:1789760702.417396:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.417549:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148288 00000100:00100000:3.0:1789760702.417559:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148288 00000100:00100000:3.0:1789760702.417613:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683614469752 time 683668218719 state 0x0 ref 1 00000100:00100000:3.0:1789760702.417627:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683668479135 time 683668218719 state 0x1 ref 1 00000100:00100000:3.0:1789760702.417660:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683668479135 time 683732962907 state 0x0 ref 1 00000100:00100000:3.0:1789760702.417673:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507148288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.417754:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16703488, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.417764:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.417781:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760702.417785:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.417792:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.417811:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.501642:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971398 is committed 00002000:00010000:3.0:1789760702.501761:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16703488 -> 16707584 00010000:00010000:3.0:1789760702.501778:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501790:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501800:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501808:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.501813:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501821:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501844:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.501852:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16703488->16707583] (req 16703488->16707583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.501961:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507148288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 84285us (84569us total) trans 4294971398 rc 0/0 00000100:00100000:3.0:1789760702.501971:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4108 00000100:00100000:3.0:1789760702.501982:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683668479135 time 683732962907 state 0x2 ref 0 00000100:00100000:1.0:1789760702.506820:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.506902:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148416 00000100:00100000:3.0:1789760702.506913:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148416 00000100:00100000:3.0:1789760702.506974:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683668479135 time 683732962907 state 0x0 ref 1 00000100:00100000:3.0:1789760702.507003:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683733223323 time 683732962907 state 0x1 ref 1 00000100:00100000:3.0:1789760702.507037:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683733223323 time 683822340563 state 0x0 ref 1 00000100:00100000:3.0:1789760702.507055:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.507114:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16707584, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.507124:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.507140:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760702.507145:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.507151:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.507166:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.603482:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971399 is committed 00002000:00010000:3.0:1789760702.603482:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16707584 -> 16711680 00010000:00010000:3.0:1789760702.603503:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603709:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603721:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603729:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.603734:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603743:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603762:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.603770:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16707584->16711679] (req 16707584->16711679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.603915:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96856us (97099us total) trans 4294971399 rc 0/0 00000100:00100000:3.0:1789760702.603930:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4109 00000100:00100000:3.0:1789760702.603940:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683733223323 time 683822340563 state 0x2 ref 0 00000100:00100000:1.0:1789760702.609583:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.609771:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148544 00000100:00100000:3.0:1789760702.609781:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148544 00000100:00100000:2.0:1789760702.610069:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683733223323 time 683822340563 state 0x0 ref 1 00000100:00100000:2.0:1789760702.610096:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683822600979 time 683822340563 state 0x1 ref 1 00000100:00100000:2.0:1789760702.610161:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683822600979 time 683925463986 state 0x0 ref 1 00000100:00100000:2.0:1789760702.610174:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507148544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760702.610217:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16711680, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:2.0:1789760702.610227:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.610242:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760702.610247:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.610253:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.610268:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.644784:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971400 is committed 00002000:00010000:0.0:1789760702.646989:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16711680 -> 16715776 00010000:00010000:0.0:1789760702.647013:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647028:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647038:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647046:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760702.647052:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647061:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647094:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647103:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6bcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16711680->16715775] (req 16711680->16715775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.647322:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507148544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37146us (37745us total) trans 4294971400 rc 0/0 00000100:00100000:0.0:1789760702.647334:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4110 00000100:00100000:0.0:1789760702.647354:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683822600979 time 683925463986 state 0x2 ref 0 00000100:00100000:0.0:1789760702.654530:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.654982:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148672 00000100:00100000:3.0:1789760702.654992:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148672 00000100:00100000:3.0:1789760702.655053:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683822600979 time 683925463986 state 0x0 ref 1 00000100:00100000:3.0:1789760702.655069:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683925724402 time 683925463986 state 0x1 ref 1 00000100:00100000:3.0:1789760702.655111:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683925724402 time 683970414090 state 0x0 ref 1 00000100:00100000:3.0:1789760702.655240:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.655290:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16715776, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.655300:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.655315:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760702.655319:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.655326:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.655340:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.683805:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760702.684009:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538254976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760702.684090:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.685536:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538254976 00000100:00100000:3.0:1789760702.685547:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538254976 00000100:00100000:3.0:1789760702.685614:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789760702.685625:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538254976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760702.685671:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760702.685760:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538254976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 137us (1668us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.685767:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00080000:3.0:1789760702.685808:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760702.688306:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760702.688332:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234628000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538255104:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760702.689497:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.689585:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255104 00000100:00100000:3.0:1789760702.689593:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255104 00000100:00100000:3.0:1789760702.690448:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:3.0:1789760702.690462:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538255104:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760702.690491:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760702.690554:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538255104:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (1059us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.690561:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1789760702.690747:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760702.690762:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e97b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538255232:0@lo:400:kworker.0 00000100:00100000:0.0:1789760702.690804:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760702.691438:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255232 00000100:00100000:0.0:1789760702.691446:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255232 00000100:00100000:0.0:1789760702.691483:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760702.691491:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538255232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760702.691535:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538255232:12345-0@lo:400:kworker.0 Request processed in 45us (731us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760702.691542:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760702.691585:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e97b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538255232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.691797:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538255744:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692128:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692158:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256256:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692196:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692235:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234628000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538255104:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760702.692251:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1789760702.692256:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538255616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692282:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692301:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538256128:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692322:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692339:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692371:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692398:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538255488:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692422:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692440:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538256000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692460:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692478:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538256512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692499:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692519:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1789760702.692525:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538255360:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692545:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692563:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538255872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692584:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692600:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538256384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692621:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.692654:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255744 00000100:00100000:3.0:1789760702.692659:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255744 00000100:00100000:3.0:1789760702.692689:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255488 00000100:00100000:3.0:1789760702.692718:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255488 00000100:00100000:3.0:1789760702.692737:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760702.692743:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538255744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692788:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538255744:12345-0@lo:400:kworker.0 Request processed in 44us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.692794:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760702.692808:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760702.692814:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538255488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692859:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538255488:12345-0@lo:400:kworker.0 Request processed in 45us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.692865:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760702.692883:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256256 00000100:00100000:3.0:1789760702.692887:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256256 00000100:00100000:3.0:1789760702.692913:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256128 00000100:00100000:3.0:1789760702.692916:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256128 00000100:00100000:3.0:1789760702.692936:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256640 00000100:00100000:3.0:1789760702.692938:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256640 00000100:00100000:3.0:1789760702.692953:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:3.0:1789760702.692959:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.692990:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256256:12345-0@lo:400:kworker.0 Request processed in 30us (794us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.692996:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:3.0:1789760702.693008:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789760702.693013:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538256128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693040:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538256128:12345-0@lo:400:kworker.0 Request processed in 27us (717us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693046:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789760702.693057:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789760702.693063:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693100:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256640:12345-0@lo:400:kworker.0 Request processed in 38us (728us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693107:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789760702.693124:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255616 00000100:00100000:3.0:1789760702.693127:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255616 00000100:00100000:3.0:1789760702.693145:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256000 00000100:00100000:3.0:1789760702.693147:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256000 00000100:00100000:3.0:1789760702.693159:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256512 00000100:00100000:3.0:1789760702.693161:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256512 00000100:00100000:3.0:1789760702.693173:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255360 00000100:00100000:3.0:1789760702.693176:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255360 00000100:00100000:3.0:1789760702.693184:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538255872 00000100:00100000:3.0:1789760702.693187:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538255872 00000100:00100000:3.0:1789760702.693198:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256384 00000100:00100000:3.0:1789760702.693200:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256384 00000100:00100000:3.0:1789760702.693214:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 812 00000100:00100000:3.0:1789760702.693220:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538255616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693251:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538255616:12345-0@lo:400:kworker.0 Request processed in 32us (968us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693257:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 812 00000100:00100000:3.0:1789760702.693269:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 813 00000100:00100000:3.0:1789760702.693275:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538256000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693303:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538256000:12345-0@lo:400:kworker.0 Request processed in 29us (842us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693308:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 813 00000100:00100000:3.0:1789760702.693318:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 814 00000100:00100000:3.0:1789760702.693323:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538256512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693350:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538256512:12345-0@lo:400:kworker.0 Request processed in 28us (850us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693356:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 814 00000100:00100000:3.0:1789760702.693367:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 815 00000100:00100000:3.0:1789760702.693374:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538255360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693400:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538255360:12345-0@lo:400:kworker.0 Request processed in 27us (853us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693405:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 815 00000100:00100000:3.0:1789760702.693415:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 816 00000100:00100000:3.0:1789760702.693421:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538255872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693446:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538255872:12345-0@lo:400:kworker.0 Request processed in 27us (861us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693452:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 816 00000100:00100000:3.0:1789760702.693467:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 817 00000100:00100000:3.0:1789760702.693473:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538256384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.693499:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538256384:12345-0@lo:400:kworker.0 Request processed in 26us (877us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.693504:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 817 00000100:00080000:3.0:1789760702.693554:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760702.693558:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760702.693561:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760702.693563:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760702.700963:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538254976:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760702.701017:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538255360:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.701031:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538255872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.701044:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538256384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.701057:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538255488:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.701069:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538256000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.702923:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538256512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.703007:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538255616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.703020:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538256128:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.703032:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.703044:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538255744:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.703057:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256256:0@lo:400:kworker.0 00000001:00080000:0.0:1789760702.751529:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971401 is committed 00002000:00010000:3.0:1789760702.751835:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16715776 -> 16719872 00010000:00010000:3.0:1789760702.751853:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751867:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751877:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751884:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.751889:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751904:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751924:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.751932:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6bd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16715776->16719871] (req 16715776->16719871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.752024:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96783us (97504us total) trans 4294971401 rc 0/0 00000100:00100000:3.0:1789760702.752034:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4111 00000100:00100000:3.0:1789760702.752045:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683925724402 time 683970414090 state 0x2 ref 0 00000100:00100000:1.0:1789760702.756759:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.756958:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148800 00000100:00100000:3.0:1789760702.756968:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148800 00000100:00100000:3.0:1789760702.757025:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683925724402 time 683970414090 state 0x0 ref 1 00000100:00100000:3.0:1789760702.757038:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683970674506 time 683970414090 state 0x1 ref 1 00000100:00100000:3.0:1789760702.757071:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683970674506 time 684072374385 state 0x0 ref 1 00000100:00100000:3.0:1789760702.757093:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.757148:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16719872, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.757157:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.757174:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760702.757178:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.757251:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.757267:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.833933:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971402 is committed 00002000:00010000:3.0:1789760702.834252:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16719872 -> 16723968 00010000:00010000:3.0:1789760702.834274:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834287:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834297:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834304:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760702.834309:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834318:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834338:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.834347:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16719872->16723967] (req 16719872->16723967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.834443:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507148800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77346us (77688us total) trans 4294971402 rc 0/0 00000100:00100000:3.0:1789760702.834453:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4112 00000100:00100000:3.0:1789760702.834463:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683970674506 time 684072374385 state 0x2 ref 0 00000100:00100000:1.0:1789760702.840582:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760702.841864:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507148928 00000100:00100000:0.0:1789760702.841875:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507148928 00000100:00100000:0.0:1789760702.841930:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 683970674506 time 684072374385 state 0x0 ref 1 00000100:00100000:0.0:1789760702.841946:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684072634801 time 684072374385 state 0x1 ref 1 00000100:00100000:0.0:1789760702.841976:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684072634801 time 684157279701 state 0x0 ref 1 00000100:00100000:0.0:1789760702.841988:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507148928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760702.842032:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16723968, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:0.0:1789760702.842043:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.842057:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760702.842061:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.842067:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.842090:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.937481:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16723968 -> 16728064 00010000:00010000:0.0:1789760702.937501:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937514:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937533:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937540:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760702.937544:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937553:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937578:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.937585:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6be0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16723968->16728063] (req 16723968->16728063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.937688:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507148928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 95695us (97108us total) trans 4294971403 rc 0/0 00000100:00100000:0.0:1789760702.937728:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4113 00000100:00100000:0.0:1789760702.937738:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684072634801 time 684157279701 state 0x2 ref 0 00000001:00080000:3.0:1789760702.938005:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971403 is committed 00000100:00100000:1.0:1789760702.956770:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760702.956995:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149056 00000100:00100000:3.0:1789760702.957006:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149056 00000100:00100000:3.0:1789760702.957065:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684072634801 time 684157279701 state 0x0 ref 1 00000100:00100000:3.0:1789760702.957090:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684157540117 time 684157279701 state 0x1 ref 1 00000100:00100000:3.0:1789760702.957127:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684157540117 time 684272430451 state 0x0 ref 1 00000100:00100000:3.0:1789760702.957139:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507149056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760702.957199:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16728064, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:3.0:1789760702.957208:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.957224:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760702.957229:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.957235:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.957250:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760703.002578:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16728064 -> 16732160 00010000:00010000:3.0:1789760703.002599:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002611:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002620:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002626:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.002631:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002784:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002804:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.002812:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6be7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16728064->16732159] (req 16728064->16732159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.002905:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507149056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45763us (46140us total) trans 4294971404 rc 0/0 00000100:00100000:3.0:1789760703.002915:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4114 00000100:00100000:3.0:1789760703.002925:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684157540117 time 684272430451 state 0x2 ref 0 00000100:00100000:3.0:1789760703.006822:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760703.007006:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149184 00000100:00100000:1.0:1789760703.007017:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149184 00000100:00100000:1.0:1789760703.007101:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684157540117 time 684272430451 state 0x0 ref 1 00000100:00100000:1.0:1789760703.007116:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684272690867 time 684272430451 state 0x1 ref 1 00000100:00100000:1.0:1789760703.007144:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684272690867 time 684322447495 state 0x0 ref 1 00000100:00100000:1.0:1789760703.007157:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507149184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760703.007217:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16732160, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8007 00010000:00010000:1.0:1789760703.007226:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.007242:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760703.007247:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.007254:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.007269:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760703.007916:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760703.010441:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971404 is committed 00002000:00010000:0.0:1789760703.112758:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16732160 -> 16736256 00002000:00010000:0.0:1789760703.112767:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760701 -> 1789760702 00002000:00010000:0.0:1789760703.112770:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760701 -> 1789760702 00010000:00010000:0.0:1789760703.112782:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112794:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112802:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112810:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760703.112816:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112841:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112862:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.112869:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6bee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16732160->16736255] (req 16732160->16736255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.112971:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507149184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 105800us (106143us total) trans 4294971405 rc 0/0 00000100:00100000:0.0:1789760703.112982:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4115 00000100:00100000:0.0:1789760703.112990:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684272690867 time 684322447495 state 0x2 ref 0 00000001:00080000:0.0:1789760703.113030:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971405 is committed 00000100:00100000:1.0:1789760703.118891:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760703.120963:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149312 00000100:00100000:0.0:1789760703.120978:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149312 00000100:00100000:0.0:1789760703.121041:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684272690867 time 684322447495 state 0x0 ref 1 00000100:00100000:0.0:1789760703.121055:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684322707911 time 684322447495 state 0x1 ref 1 00000100:00100000:0.0:1789760703.121099:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684322707911 time 684436402081 state 0x0 ref 1 00000100:00100000:0.0:1789760703.121178:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507149312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760703.121230:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16736256, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:0.0:1789760703.121238:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.121256:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760703.121260:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.121267:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.121280:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.241548:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971406 is committed 00002000:00010000:3.0:1789760703.241727:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16736256 -> 16740352 00010000:00010000:3.0:1789760703.241741:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241752:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241760:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241766:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.241771:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241778:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241796:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.241803:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b6bf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16736256->16740351] (req 16736256->16740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.241910:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507149312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 120796us (123025us total) trans 4294971406 rc 0/0 00000100:00100000:3.0:1789760703.241919:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4116 00000100:00100000:3.0:1789760703.241928:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684322707911 time 684436402081 state 0x2 ref 0 00000100:00100000:1.0:1789760703.246560:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760703.246686:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149440 00000100:00100000:3.0:1789760703.246721:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149440 00000100:00100000:3.0:1789760703.246772:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684322707911 time 684436402081 state 0x0 ref 1 00000100:00100000:3.0:1789760703.246786:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684436662497 time 684436402081 state 0x1 ref 1 00000100:00100000:3.0:1789760703.246813:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684436662497 time 684562117102 state 0x0 ref 1 00000100:00100000:3.0:1789760703.246839:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507149440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760703.246889:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16740352, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:3.0:1789760703.246897:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.246913:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760703.246917:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.246922:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.246939:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.345064:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971407 is committed 00002000:00010000:2.0:1789760703.346253:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16740352 -> 16744448 00010000:00010000:2.0:1789760703.346272:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346284:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346293:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346305:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760703.346310:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346319:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346339:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.346347:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6bfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16740352->16744447] (req 16740352->16744447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.346454:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507149440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 99612us (99897us total) trans 4294971407 rc 0/0 00000100:00100000:2.0:1789760703.346465:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4117 00000100:00100000:2.0:1789760703.346474:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684436662497 time 684562117102 state 0x2 ref 0 00000100:00100000:1.0:1789760703.350636:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760703.350945:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149568 00000100:00100000:0.0:1789760703.350955:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149568 00000100:00100000:1.0:1789760703.351097:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684436662497 time 684562117102 state 0x0 ref 1 00000100:00100000:1.0:1789760703.351173:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684562377518 time 684562117102 state 0x1 ref 1 00000100:00100000:1.0:1789760703.351297:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684562377518 time 684666599228 state 0x0 ref 1 00000100:00100000:1.0:1789760703.351311:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507149568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760703.351363:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16744448, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:1.0:1789760703.351373:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.351388:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760703.351393:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.351399:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.351413:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.388315:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760703.430334:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16744448 -> 16748544 00010000:00010000:3.0:1789760703.430356:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430369:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430378:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430385:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.430389:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430398:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430414:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.430422:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6c03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16744448->16748543] (req 16744448->16748543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.430512:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507149568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79199us (79883us total) trans 4294971408 rc 0/0 00000100:00100000:3.0:1789760703.430522:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4118 00000100:00100000:3.0:1789760703.430531:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684562377518 time 684666599228 state 0x2 ref 0 00000001:00080000:1.0:1789760703.431189:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971408 is committed 00000100:00100000:1.0:1789760703.434585:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760703.434683:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149696 00000100:00100000:3.0:1789760703.434726:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149696 00000100:00100000:2.0:1789760703.434813:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684562377518 time 684666599228 state 0x0 ref 1 00000100:00100000:2.0:1789760703.434840:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684666859644 time 684666599228 state 0x1 ref 1 00000100:00100000:2.0:1789760703.434869:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684666859644 time 684750172383 state 0x0 ref 1 00000100:00100000:2.0:1789760703.434884:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507149696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760703.434932:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16748544, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:2.0:1789760703.434942:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.434957:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760703.434962:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.434968:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.434983:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.482389:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971409 is committed 00002000:00010000:2.0:1789760703.482480:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16748544 -> 16752640 00010000:00010000:2.0:1789760703.482496:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482509:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482529:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482537:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760703.482541:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482551:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482571:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.482579:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6c0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16748544->16752639] (req 16748544->16752639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.483275:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507149696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48387us (48691us total) trans 4294971409 rc 0/0 00000100:00100000:0.0:1789760703.483286:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4119 00000100:00100000:0.0:1789760703.483295:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684666859644 time 684750172383 state 0x2 ref 0 00000100:00100000:0.0:1789760703.492031:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760703.492928:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149824 00000100:00100000:2.0:1789760703.492939:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149824 00000100:00100000:2.0:1789760703.492999:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684666859644 time 684750172383 state 0x0 ref 1 00000100:00100000:2.0:1789760703.493015:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684750432799 time 684750172383 state 0x1 ref 1 00000100:00100000:2.0:1789760703.493265:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684750432799 time 684808410955 state 0x0 ref 1 00000100:00100000:2.0:1789760703.493281:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507149824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760703.493332:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16752640, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:2.0:1789760703.493341:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.493356:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760703.493361:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.493367:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.493382:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.536192:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971410 is committed 00002000:00010000:3.0:1789760703.536954:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16752640 -> 16756736 00010000:00010000:3.0:1789760703.536972:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.536997:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.537007:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.537015:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.537020:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.537030:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.537052:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.537066:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b6c11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16752640->16756735] (req 16752640->16756735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.537176:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507149824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43893us (45149us total) trans 4294971410 rc 0/0 00000100:00100000:3.0:1789760703.537187:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4120 00000100:00100000:3.0:1789760703.537197:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684750432799 time 684808410955 state 0x2 ref 0 00000100:00100000:1.0:1789760703.550834:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760703.551006:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507149952 00000100:00100000:3.0:1789760703.551017:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507149952 00000100:00100000:3.0:1789760703.551086:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684750432799 time 684808410955 state 0x0 ref 1 00000100:00100000:3.0:1789760703.551103:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684808671371 time 684808410955 state 0x1 ref 1 00000100:00100000:3.0:1789760703.551139:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684808671371 time 684866442998 state 0x0 ref 1 00000100:00100000:3.0:1789760703.551153:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507149952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760703.551213:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16756736, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:3.0:1789760703.551222:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.551238:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760703.551242:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.551250:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.551279:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.612273:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971411 is committed 00002000:00010000:1.0:1789760703.612427:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16756736 -> 16760832 00010000:00010000:1.0:1789760703.612444:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612459:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612468:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612481:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760703.612485:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612494:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612525:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.612533:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6c18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16756736->16760831] (req 16756736->16760831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.612628:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507149952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61472us (61800us total) trans 4294971411 rc 0/0 00000100:00100000:1.0:1789760703.613058:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4121 00000100:00100000:1.0:1789760703.613068:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684808671371 time 684866442998 state 0x2 ref 0 00000100:00100000:1.0:1789760703.621442:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760703.622093:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150080 00000100:00100000:0.0:1789760703.622105:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150080 00000100:00100000:0.0:1789760703.622167:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684808671371 time 684866442998 state 0x0 ref 1 00000100:00100000:0.0:1789760703.622183:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684866703414 time 684866442998 state 0x1 ref 1 00000100:00100000:0.0:1789760703.622218:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684866703414 time 684937521050 state 0x0 ref 1 00000100:00100000:0.0:1789760703.622231:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760703.622281:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16760832, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:0.0:1789760703.622290:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.622306:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760703.622311:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.622316:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.622330:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.682405:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16760832 -> 16764928 00010000:00010000:1.0:1789760703.682424:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682437:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682452:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682460:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760703.682464:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682472:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682492:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.682501:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6c1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16760832->16764927] (req 16760832->16764927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.682593:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507150080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60358us (61154us total) trans 4294971412 rc 0/0 00000100:00100000:1.0:1789760703.682604:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4122 00000100:00100000:1.0:1789760703.682613:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684866703414 time 684937521050 state 0x2 ref 0 00000001:00080000:3.0:1789760703.687068:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971412 is committed 00000100:00100000:1.0:1789760703.704336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760703.705901:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150208 00000100:00100000:0.0:1789760703.705914:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150208 00000100:00100000:0.0:1789760703.705976:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684866703414 time 684937521050 state 0x0 ref 1 00000100:00100000:0.0:1789760703.706065:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684937781466 time 684937521050 state 0x1 ref 1 00000100:00100000:0.0:1789760703.706211:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684937781466 time 685021409323 state 0x0 ref 1 00000100:00100000:0.0:1789760703.706229:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760703.706277:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16764928, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:0.0:1789760703.706288:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.706305:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760703.706309:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.706316:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.706334:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.711006:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760703.711090:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760703.711624:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538256896:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760703.712153:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760703.713325:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256768 00000100:00100000:0.0:1789760703.713336:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256768 00000100:00100000:0.0:1789760703.713377:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538256896 00000100:00100000:0.0:1789760703.713380:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538256896 00000100:00100000:0.0:1789760703.713394:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1789760703.713401:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760703.713424:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3841982464 avail 3828350976 00000100:00100000:0.0:1789760703.713482:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538256768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (2393us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760703.713488:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1789760703.713504:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789760703.713510:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538256896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760703.714841:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3841982464 avail 3828350976 00000100:00100000:0.0:1789760703.714906:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538256896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1394us (2752us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760703.714916:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789760703.718093:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538256768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760703.719263:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538256896:0@lo:13:osp-pre-0-1.0 00000001:00080000:1.0:1789760703.770771:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971413 is committed 00002000:00010000:3.0:1789760703.770960:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16764928 -> 16769024 00010000:00010000:3.0:1789760703.771043:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771056:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771065:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771082:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.771086:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771095:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771185:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.771195:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b6c26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16764928->16769023] (req 16764928->16769023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.771328:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65099us (66996us total) trans 4294971413 rc 0/0 00000100:00100000:3.0:1789760703.771340:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4123 00000100:00100000:3.0:1789760703.771350:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684937781466 time 685021409323 state 0x2 ref 0 00000100:00100000:2.0:1789760703.776858:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760703.777173:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150336 00000100:00100000:1.0:1789760703.777183:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150336 00000100:00100000:1.0:1789760703.777236:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 684937781466 time 685021409323 state 0x0 ref 1 00000100:00100000:1.0:1789760703.777252:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685021669739 time 685021409323 state 0x1 ref 1 00000100:00100000:1.0:1789760703.777310:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685021669739 time 685092613311 state 0x0 ref 1 00000100:00100000:1.0:1789760703.777323:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760703.777372:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16769024, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:1.0:1789760703.777380:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.777396:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760703.777401:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.777407:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.777422:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.813029:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16769024 -> 16773120 00010000:00010000:1.0:1789760703.813050:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.813063:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.813083:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.813092:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760703.813097:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.813114:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.813129:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971414 is committed 00010000:00010000:1.0:1789760703.813134:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.813141:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16769024->16773119] (req 16769024->16773119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.813232:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35906us (36380us total) trans 4294971414 rc 0/0 00000100:00100000:1.0:1789760703.813242:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4124 00000100:00100000:1.0:1789760703.813252:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685021669739 time 685092613311 state 0x2 ref 0 00000100:00100000:1.0:1789760703.819951:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760703.820081:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150464 00000100:00100000:0.0:1789760703.820094:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150464 00000100:00100000:0.0:1789760703.820207:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685021669739 time 685092613311 state 0x0 ref 1 00000100:00100000:0.0:1789760703.820223:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685092873727 time 685092613311 state 0x1 ref 1 00000100:00100000:0.0:1789760703.820255:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685092873727 time 685135557988 state 0x0 ref 1 00000100:00100000:0.0:1789760703.820267:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507150464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760703.820315:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16773120, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:0.0:1789760703.820325:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.820340:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760703.820345:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.820351:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.820370:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.874482:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971415 is committed 00002000:00010000:1.0:1789760703.874802:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16773120 -> 16777216 00010000:00010000:1.0:1789760703.874817:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.874959:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.874971:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.874978:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760703.874984:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.874994:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.875015:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.875023:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b6c34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16773120->16777215] (req 16773120->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.875167:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507150464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54897us (55220us total) trans 4294971415 rc 0/0 00000100:00100000:1.0:1789760703.875188:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4125 00000100:00100000:1.0:1789760703.875198:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685092873727 time 685135557988 state 0x2 ref 0 00000100:00100000:2.0:1789760703.878652:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760703.879238:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150592 00000100:00100000:3.0:1789760703.879249:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150592 00000100:00100000:2.0:1789760703.880464:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685092873727 time 685135557988 state 0x0 ref 1 00000100:00100000:2.0:1789760703.880482:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685135818404 time 685135557988 state 0x1 ref 1 00000100:00100000:2.0:1789760703.880528:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685135818404 time 685195830426 state 0x0 ref 1 00000100:00100000:2.0:1789760703.880542:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760703.880587:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16777216, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8007 00010000:00010000:2.0:1789760703.880597:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.880612:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760703.880617:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.880623:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.880637:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.916024:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971416 is committed 00002000:00010000:2.0:1789760703.916143:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16777216 -> 16781312 00002000:00010000:2.0:1789760703.916148:0:14616:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:2.0:1789760703.916160:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916287:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916298:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916306:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760703.916310:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916320:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916342:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.916350:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6c3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->16781311] (req 16777216->16781311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.916495:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507150592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35950us (37846us total) trans 4294971416 rc 0/0 00000100:00100000:2.0:1789760703.916507:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4126 00000100:00100000:2.0:1789760703.916516:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685135818404 time 685195830426 state 0x2 ref 0 00000100:00100000:2.0:1789760703.920534:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760703.920613:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150720 00000100:00100000:3.0:1789760703.920622:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150720 00000100:00100000:3.0:1789760703.920679:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685135818404 time 685195830426 state 0x0 ref 1 00000100:00100000:3.0:1789760703.920726:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685196090842 time 685195830426 state 0x1 ref 1 00000100:00100000:3.0:1789760703.920916:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685196090842 time 685236217140 state 0x0 ref 1 00000100:00100000:3.0:1789760703.920930:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760703.920975:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16781312, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8807 00010000:00010000:3.0:1789760703.920984:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.921000:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760703.921005:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.921011:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.921025:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.966566:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971417 is committed 00002000:00010000:3.0:1789760703.966675:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16781312 -> 16785408 00010000:00010000:3.0:1789760703.966717:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966731:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966740:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966844:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760703.966850:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966859:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966880:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.966887:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16781312->16785407] (req 16781312->16785407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.967046:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46114us (46528us total) trans 4294971417 rc 0/0 00000100:00100000:3.0:1789760703.967059:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4127 00000100:00100000:3.0:1789760703.967079:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685196090842 time 685236217140 state 0x2 ref 0 00000100:00100000:2.0:1789760703.972133:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760703.974666:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150848 00000100:00100000:1.0:1789760703.974680:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150848 00000100:00100000:1.0:1789760703.974833:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685196090842 time 685236217140 state 0x0 ref 1 00000100:00100000:1.0:1789760703.974849:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685236477556 time 685236217140 state 0x1 ref 1 00000100:00100000:1.0:1789760703.974912:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685236477556 time 685290214607 state 0x0 ref 1 00000100:00100000:1.0:1789760703.974925:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760703.974969:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16785408, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8807 00010000:00010000:1.0:1789760703.974979:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.974995:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760703.974999:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.975005:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.975018:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.016414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971418 is committed 00002000:00010000:1.0:1789760704.016579:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16785408 -> 16789504 00010000:00010000:1.0:1789760704.016596:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016610:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016619:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016627:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760704.016633:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016642:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016662:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.016670:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6c49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16785408->16789503] (req 16785408->16789503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.016880:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41952us (44810us total) trans 4294971418 rc 0/0 00000100:00100000:1.0:1789760704.016890:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4128 00000100:00100000:1.0:1789760704.016900:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685236477556 time 685290214607 state 0x2 ref 0 00000100:00100000:2.0:1789760704.020593:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760704.020817:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507150976 00000100:00100000:0.0:1789760704.020826:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507150976 00000100:00100000:0.0:1789760704.020895:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685236477556 time 685290214607 state 0x0 ref 1 00000100:00100000:0.0:1789760704.020913:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685290475023 time 685290214607 state 0x1 ref 1 00000100:00100000:0.0:1789760704.020938:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685290475023 time 685336241697 state 0x0 ref 1 00000100:00100000:0.0:1789760704.020951:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507150976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760704.020998:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16789504, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x8807 00010000:00010000:0.0:1789760704.021008:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.021023:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760704.021028:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.021036:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.021051:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.079932:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16789504 -> 16793600 00002000:00010000:0.0:1789760704.079941:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:0.0:1789760704.079945:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:0.0:1789760704.079957:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.079971:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.079980:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.079987:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760704.079992:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.080001:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.080020:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.080028:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16789504->16793599] (req 16789504->16793599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.080136:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507150976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59182us (59548us total) trans 4294971419 rc 0/0 00000100:00100000:0.0:1789760704.080146:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4129 00000100:00100000:0.0:1789760704.080156:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685290475023 time 685336241697 state 0x2 ref 0 00000001:00080000:1.0:1789760704.080624:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971419 is committed 00000100:00100000:2.0:1789760704.084232:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.088098:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151104 00000100:00100000:3.0:1789760704.088113:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151104 00000100:00100000:3.0:1789760704.088177:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685290475023 time 685336241697 state 0x0 ref 1 00000100:00100000:3.0:1789760704.088191:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685336502113 time 685336241697 state 0x1 ref 1 00000100:00100000:3.0:1789760704.088227:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685336502113 time 685403530697 state 0x0 ref 1 00000100:00100000:3.0:1789760704.088240:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240b8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.088286:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16793600, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.088295:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.088310:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760704.088315:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.088321:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.088334:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760704.135105:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16793600 -> 16797696 00010000:00010000:3.0:1789760704.135124:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135137:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135148:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135156:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.135161:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135170:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135189:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.135197:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16793600->16797695] (req 16793600->16797695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.135286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971420 is committed 00000100:00100000:3.0:1789760704.135298:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240b8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47056us (51071us total) trans 4294971420 rc 0/0 00000100:00100000:3.0:1789760704.135310:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4130 00000100:00100000:3.0:1789760704.135320:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685336502113 time 685403530697 state 0x2 ref 0 00000100:00100000:3.0:1789760704.140361:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760704.140579:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151232 00000100:00100000:1.0:1789760704.140588:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151232 00000100:00100000:1.0:1789760704.140649:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685336502113 time 685403530697 state 0x0 ref 1 00000100:00100000:1.0:1789760704.140666:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685403791113 time 685403530697 state 0x1 ref 1 00000100:00100000:1.0:1789760704.140733:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685403791113 time 685456035826 state 0x0 ref 1 00000100:00100000:1.0:1789760704.140748:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507151232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760704.140802:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16797696, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:1.0:1789760704.140811:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.140830:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760704.140835:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.140840:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.140853:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.178574:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971421 is committed 00002000:00010000:0.0:1789760704.178657:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16797696 -> 16801792 00010000:00010000:0.0:1789760704.178672:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178685:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178858:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178865:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760704.178869:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178876:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178894:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.178899:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b6c5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16797696->16801791] (req 16797696->16801791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.179030:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507151232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38280us (38674us total) trans 4294971421 rc 0/0 00000100:00100000:0.0:1789760704.179039:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4131 00000100:00100000:0.0:1789760704.179048:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685403791113 time 685456035826 state 0x2 ref 0 00000100:00100000:1.0:1789760704.183822:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760704.184204:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151360 00000100:00100000:0.0:1789760704.184214:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151360 00000100:00100000:0.0:1789760704.184263:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685403791113 time 685456035826 state 0x0 ref 1 00000100:00100000:0.0:1789760704.184277:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685456296242 time 685456035826 state 0x1 ref 1 00000100:00100000:0.0:1789760704.184304:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685456296242 time 685499607039 state 0x0 ref 1 00000100:00100000:0.0:1789760704.184315:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760704.184362:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16801792, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:0.0:1789760704.184375:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.184391:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760704.184397:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.184404:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.184418:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.211268:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971422 is committed 00002000:00010000:3.0:1789760704.211356:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16801792 -> 16805888 00010000:00010000:3.0:1789760704.211374:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211389:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211400:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211408:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.211413:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211422:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211442:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.211449:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6c65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16801792->16805887] (req 16801792->16805887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.211971:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27649us (28151us total) trans 4294971422 rc 0/0 00000100:00100000:3.0:1789760704.211988:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4132 00000100:00100000:3.0:1789760704.212000:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685456296242 time 685499607039 state 0x2 ref 0 00000100:00100000:3.0:1789760704.217419:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760704.217613:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151488 00000100:00100000:1.0:1789760704.217680:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151488 00000100:00100000:1.0:1789760704.217937:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685456296242 time 685499607039 state 0x0 ref 1 00000100:00100000:1.0:1789760704.217954:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685499867455 time 685499607039 state 0x1 ref 1 00000100:00100000:1.0:1789760704.217990:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685499867455 time 685533293141 state 0x0 ref 1 00000100:00100000:1.0:1789760704.218002:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507151488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760704.218051:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16805888, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:1.0:1789760704.218060:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.218091:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760704.218096:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.218104:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.218120:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.219898:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538257024:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760704.219972:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760704.220005:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257024 00000100:00100000:3.0:1789760704.220014:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257024 00000100:00100000:3.0:1789760704.220075:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760704.220087:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538257024:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760704.220125:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760704.220180:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538257024:12345-0@lo:41:osp-pre-1-0.0 Request processed in 95us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760704.220189:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760704.220301:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538257024:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789760704.250163:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971423 is committed 00002000:00010000:3.0:1789760704.250346:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16805888 -> 16809984 00010000:00010000:3.0:1789760704.250362:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250375:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250383:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250391:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.250396:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250406:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250427:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.250434:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6c6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16805888->16809983] (req 16805888->16809983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.250602:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507151488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32598us (33188us total) trans 4294971423 rc 0/0 00000100:00100000:3.0:1789760704.250613:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4133 00000100:00100000:3.0:1789760704.250623:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685499867455 time 685533293141 state 0x2 ref 0 00000100:00100000:2.0:1789760704.253981:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760704.254107:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151616 00000100:00100000:0.0:1789760704.254117:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151616 00000100:00100000:0.0:1789760704.254175:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685499867455 time 685533293141 state 0x0 ref 1 00000100:00100000:0.0:1789760704.254191:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685533553557 time 685533293141 state 0x1 ref 1 00000100:00100000:0.0:1789760704.254317:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685533553557 time 685569619364 state 0x0 ref 1 00000100:00100000:0.0:1789760704.254332:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507151616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760704.254380:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16809984, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:0.0:1789760704.254389:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.254405:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760704.254409:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.254415:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.254428:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.285477:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971424 is committed 00002000:00010000:0.0:1789760704.285575:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16809984 -> 16814080 00010000:00010000:0.0:1789760704.285592:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285602:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285611:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285623:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760704.285628:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285637:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285658:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.285666:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6c73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16809984->16814079] (req 16809984->16814079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.285971:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507151616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31636us (31994us total) trans 4294971424 rc 0/0 00000100:00100000:0.0:1789760704.285982:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4134 00000100:00100000:0.0:1789760704.285991:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685533553557 time 685569619364 state 0x2 ref 0 00000100:00100000:1.0:1789760704.290321:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.290409:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151744 00000100:00100000:3.0:1789760704.290418:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151744 00000100:00100000:3.0:1789760704.290473:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685533553557 time 685569619364 state 0x0 ref 1 00000100:00100000:3.0:1789760704.290489:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685569879780 time 685569619364 state 0x1 ref 1 00000100:00100000:3.0:1789760704.290520:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685569879780 time 685605823398 state 0x0 ref 1 00000100:00100000:3.0:1789760704.290533:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.290576:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16814080, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.290586:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.290601:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760704.290606:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.290611:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.290625:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.337665:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971425 is committed 00002000:00010000:3.0:1789760704.337992:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16814080 -> 16818176 00010000:00010000:3.0:1789760704.338008:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338021:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338030:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338038:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.338043:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338053:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338085:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.338093:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16814080->16818175] (req 16814080->16818175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.338194:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507151744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47657us (47877us total) trans 4294971425 rc 0/0 00000100:00100000:3.0:1789760704.338207:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4135 00000100:00100000:3.0:1789760704.338217:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685569879780 time 685605823398 state 0x2 ref 0 00000100:00100000:1.0:1789760704.343881:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.344669:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507151872 00000100:00100000:3.0:1789760704.344680:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507151872 00000100:00100000:3.0:1789760704.344766:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685569879780 time 685605823398 state 0x0 ref 1 00000100:00100000:3.0:1789760704.344783:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685606083814 time 685605823398 state 0x1 ref 1 00000100:00100000:3.0:1789760704.344813:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685606083814 time 685660115999 state 0x0 ref 1 00000100:00100000:3.0:1789760704.344826:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507151872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.344874:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16818176, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.344883:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.344899:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760704.344903:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.344910:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.344930:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.402092:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971426 is committed 00002000:00010000:3.0:1789760704.402253:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16818176 -> 16822272 00010000:00010000:3.0:1789760704.402337:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402351:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402360:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402367:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.402373:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402383:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402402:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.402411:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16818176->16822271] (req 16818176->16822271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.402507:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507151872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57677us (58630us total) trans 4294971426 rc 0/0 00000100:00100000:3.0:1789760704.402666:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4136 00000100:00100000:3.0:1789760704.402676:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685606083814 time 685660115999 state 0x2 ref 0 00000100:00100000:0.0:1789760704.406612:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760704.406772:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152000 00000100:00100000:1.0:1789760704.406782:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152000 00000100:00100000:1.0:1789760704.406835:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685606083814 time 685660115999 state 0x0 ref 1 00000100:00100000:1.0:1789760704.406850:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685660376415 time 685660115999 state 0x1 ref 1 00000100:00100000:1.0:1789760704.406879:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685660376415 time 685722182169 state 0x0 ref 1 00000100:00100000:1.0:1789760704.406892:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507152000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760704.406939:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16822272, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:1.0:1789760704.406948:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.406963:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760704.406967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.406974:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.406988:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.413020:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760704.440623:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971427 is committed 00002000:00010000:2.0:1789760704.440756:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16822272 -> 16826368 00010000:00010000:2.0:1789760704.440770:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440784:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440793:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440801:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.440806:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440815:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440836:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.440844:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6c88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16822272->16826367] (req 16822272->16826367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.440933:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507152000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34038us (34325us total) trans 4294971427 rc 0/0 00000100:00100000:2.0:1789760704.440942:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4137 00000100:00100000:2.0:1789760704.440951:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685660376415 time 685722182169 state 0x2 ref 0 00000100:00100000:1.0:1789760704.445368:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.445484:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152128 00000100:00100000:3.0:1789760704.445494:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152128 00000100:00100000:3.0:1789760704.445608:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685660376415 time 685722182169 state 0x0 ref 1 00000100:00100000:3.0:1789760704.445624:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685722442585 time 685722182169 state 0x1 ref 1 00000100:00100000:3.0:1789760704.445687:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685722442585 time 685760989726 state 0x0 ref 1 00000100:00100000:3.0:1789760704.445808:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507152128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.445855:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16826368, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.445865:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.445881:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760704.445885:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.445891:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.445904:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.477741:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971428 is committed 00002000:00010000:1.0:1789760704.477971:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16826368 -> 16830464 00010000:00010000:1.0:1789760704.477987:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478002:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478011:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478019:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760704.478023:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478033:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478058:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.478076:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6c8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16826368->16830463] (req 16826368->16830463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.478327:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507152128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32517us (32963us total) trans 4294971428 rc 0/0 00000100:00100000:1.0:1789760704.478338:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4138 00000100:00100000:1.0:1789760704.478348:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685722442585 time 685760989726 state 0x2 ref 0 00000100:00100000:1.0:1789760704.482014:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.482166:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152256 00000100:00100000:3.0:1789760704.482176:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152256 00000100:00100000:3.0:1789760704.482229:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685722442585 time 685760989726 state 0x0 ref 1 00000100:00100000:3.0:1789760704.482243:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685761250142 time 685760989726 state 0x1 ref 1 00000100:00100000:3.0:1789760704.482364:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685761250142 time 685797666806 state 0x0 ref 1 00000100:00100000:3.0:1789760704.482376:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507152256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.482422:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16830464, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.482432:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.482448:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760704.482452:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.482459:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.482473:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.527588:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16830464 -> 16834560 00010000:00010000:2.0:1789760704.527609:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527622:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527632:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527640:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.527645:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527654:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527674:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.527689:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6c96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16830464->16834559] (req 16830464->16834559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.527807:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507152256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45418us (45789us total) trans 4294971429 rc 0/0 00000100:00100000:2.0:1789760704.527818:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4139 00000100:00100000:2.0:1789760704.527827:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685761250142 time 685797666806 state 0x2 ref 0 00000001:00080000:3.0:1789760704.530319:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971429 is committed 00000100:00100000:0.0:1789760704.531411:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.531516:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152384 00000100:00100000:2.0:1789760704.531527:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152384 00000100:00100000:2.0:1789760704.531727:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685761250142 time 685797666806 state 0x0 ref 1 00000100:00100000:2.0:1789760704.531743:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685797927222 time 685797666806 state 0x1 ref 1 00000100:00100000:2.0:1789760704.531778:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685797927222 time 685847081780 state 0x0 ref 1 00000100:00100000:2.0:1789760704.531791:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.531840:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16834560, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.531850:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.531867:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760704.531871:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.531878:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.531895:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.594179:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971430 is committed 00002000:00010000:2.0:1789760704.594325:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16834560 -> 16838656 00010000:00010000:2.0:1789760704.594341:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594353:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594362:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594369:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.594374:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594382:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594402:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.594411:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6c9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16834560->16838655] (req 16834560->16838655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.594503:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62710us (63094us total) trans 4294971430 rc 0/0 00000100:00100000:2.0:1789760704.594520:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4140 00000100:00100000:2.0:1789760704.594529:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685797927222 time 685847081780 state 0x2 ref 0 00000100:00100000:0.0:1789760704.603410:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760704.604008:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152512 00000100:00100000:1.0:1789760704.604019:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152512 00000100:00100000:1.0:1789760704.604090:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685797927222 time 685847081780 state 0x0 ref 1 00000100:00100000:1.0:1789760704.604106:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685847342196 time 685847081780 state 0x1 ref 1 00000100:00100000:1.0:1789760704.604141:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685847342196 time 685919443980 state 0x0 ref 1 00000100:00100000:1.0:1789760704.604154:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760704.604200:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16838656, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:1.0:1789760704.604210:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.604231:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760704.604236:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.604243:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.604258:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.633265:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971431 is committed 00002000:00010000:2.0:1789760704.633353:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16838656 -> 16842752 00010000:00010000:2.0:1789760704.633368:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633382:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633392:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633400:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.633405:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633413:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633433:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633445:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6ca4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16838656->16842751] (req 16838656->16842751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.633547:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29389us (30141us total) trans 4294971431 rc 0/0 00000100:00100000:2.0:1789760704.633558:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4141 00000100:00100000:2.0:1789760704.633573:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685847342196 time 685919443980 state 0x2 ref 0 00000100:00100000:3.0:1789760704.636962:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.640312:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152640 00000100:00100000:2.0:1789760704.640325:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152640 00000100:00100000:2.0:1789760704.640382:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685847342196 time 685919443980 state 0x0 ref 1 00000100:00100000:2.0:1789760704.640396:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685919704396 time 685919443980 state 0x1 ref 1 00000100:00100000:2.0:1789760704.640427:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685919704396 time 685955730743 state 0x0 ref 1 00000100:00100000:2.0:1789760704.640439:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507152640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.640479:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16842752, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.640488:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.640503:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760704.640508:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.640513:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.640532:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.679264:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16842752 -> 16846848 00000001:00080000:0.0:1789760704.679266:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971432 is committed 00010000:00010000:2.0:1789760704.679282:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679294:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679304:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679312:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.679317:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679327:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679346:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.679352:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16842752->16846847] (req 16842752->16846847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.679445:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507152640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39004us (42487us total) trans 4294971432 rc 0/0 00000100:00100000:2.0:1789760704.679455:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4142 00000100:00100000:2.0:1789760704.679464:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685919704396 time 685955730743 state 0x2 ref 0 00000100:00100000:0.0:1789760704.682424:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.682562:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152768 00000100:00100000:2.0:1789760704.682572:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152768 00000100:00100000:2.0:1789760704.682626:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685919704396 time 685955730743 state 0x0 ref 1 00000100:00100000:2.0:1789760704.682641:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685955991159 time 685955730743 state 0x1 ref 1 00000100:00100000:2.0:1789760704.682741:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685955991159 time 685998043903 state 0x0 ref 1 00000100:00100000:2.0:1789760704.682754:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.682803:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16846848, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.682811:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.682826:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760704.682831:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.682836:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.682851:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.706989:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971433 is committed 00002000:00010000:2.0:1789760704.707160:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16846848 -> 16850944 00010000:00010000:2.0:1789760704.707177:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707189:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707198:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707205:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.707210:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707220:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707245:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.707252:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16846848->16850943] (req 16846848->16850943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.707359:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24602us (24937us total) trans 4294971433 rc 0/0 00000100:00100000:2.0:1789760704.707374:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4143 00000100:00100000:2.0:1789760704.707383:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685955991159 time 685998043903 state 0x2 ref 0 00000100:00100000:0.0:1789760704.711018:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.711061:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507152896 00000100:00100000:2.0:1789760704.711080:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507152896 00000100:00100000:2.0:1789760704.711133:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685955991159 time 685998043903 state 0x0 ref 1 00000100:00100000:2.0:1789760704.711148:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685998304319 time 685998043903 state 0x1 ref 1 00000100:00100000:2.0:1789760704.711176:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685998304319 time 686026479822 state 0x0 ref 1 00000100:00100000:2.0:1789760704.711188:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507152896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.711234:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16850944, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.711243:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.711257:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760704.711261:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.711267:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.711281:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.737120:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760704.737142:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10406a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538257152:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760704.737203:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760704.737552:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257152 00000100:00100000:3.0:1789760704.737560:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257152 00000100:00100000:3.0:1789760704.737611:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:3.0:1789760704.737621:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538257152:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760704.737659:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760704.737744:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538257152:12345-0@lo:41:osp-pre-0-1.0 Request processed in 124us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760704.737751:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789760704.737916:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10406a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538257152:0@lo:41:osp-pre-0-1.0 00002000:00010000:2.0:1789760704.775425:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16850944 -> 16855040 00010000:00010000:2.0:1789760704.775444:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775463:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775473:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775481:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.775487:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775497:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775522:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.775530:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16850944->16855039] (req 16850944->16855039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.775625:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507152896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64433us (64610us total) trans 4294971434 rc 0/0 00000100:00100000:2.0:1789760704.775650:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4144 00000100:00100000:2.0:1789760704.775663:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685998304319 time 686026479822 state 0x2 ref 0 00000001:00080000:1.0:1789760704.775825:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971434 is committed 00000100:00100000:1.0:1789760704.780305:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.780429:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153024 00000100:00100000:3.0:1789760704.780438:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153024 00000100:00100000:3.0:1789760704.780489:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 685998304319 time 686026479822 state 0x0 ref 1 00000100:00100000:3.0:1789760704.780504:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686026740238 time 686026479822 state 0x1 ref 1 00000100:00100000:3.0:1789760704.780543:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686026740238 time 686095845671 state 0x0 ref 1 00000100:00100000:3.0:1789760704.780555:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507153024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760704.780604:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16855040, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:3.0:1789760704.780613:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.780629:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760704.780634:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.780640:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.780654:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.812559:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971435 is committed 00002000:00010000:2.0:1789760704.812741:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16855040 -> 16859136 00010000:00010000:2.0:1789760704.812757:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812769:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812778:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812785:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.812790:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812799:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812818:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.812825:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16855040->16859135] (req 16855040->16859135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.812918:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507153024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32360us (32618us total) trans 4294971435 rc 0/0 00000100:00100000:2.0:1789760704.812932:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4145 00000100:00100000:2.0:1789760704.812941:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686026740238 time 686095845671 state 0x2 ref 0 00000100:00100000:0.0:1789760704.816151:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.816226:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153152 00000100:00100000:2.0:1789760704.816235:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153152 00000100:00100000:2.0:1789760704.816285:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686026740238 time 686095845671 state 0x0 ref 1 00000100:00100000:2.0:1789760704.816299:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686096106087 time 686095845671 state 0x1 ref 1 00000100:00100000:2.0:1789760704.816336:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686096106087 time 686131631583 state 0x0 ref 1 00000100:00100000:2.0:1789760704.816348:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507153152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.816393:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16859136, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.816401:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.816416:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760704.816420:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.816426:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.816440:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.843958:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971436 is committed 00002000:00010000:3.0:1789760704.844210:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16859136 -> 16863232 00010000:00010000:3.0:1789760704.844227:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844240:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844249:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844256:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.844260:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844270:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844289:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.844297:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6cc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16859136->16863231] (req 16859136->16863231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.844533:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507153152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28181us (28385us total) trans 4294971436 rc 0/0 00000100:00100000:3.0:1789760704.844548:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4146 00000100:00100000:3.0:1789760704.844558:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686096106087 time 686131631583 state 0x2 ref 0 00000100:00100000:0.0:1789760704.848154:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.848237:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153280 00000100:00100000:2.0:1789760704.848245:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153280 00000100:00100000:2.0:1789760704.848291:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686096106087 time 686131631583 state 0x0 ref 1 00000100:00100000:2.0:1789760704.848302:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686131891999 time 686131631583 state 0x1 ref 1 00000100:00100000:2.0:1789760704.848398:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686131891999 time 686163701958 state 0x0 ref 1 00000100:00100000:2.0:1789760704.848408:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507153280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.848453:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16863232, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.848463:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.848478:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760704.848482:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.848489:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.848504:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.876994:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16863232 -> 16867328 00010000:00010000:2.0:1789760704.877013:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.877027:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.877035:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.877043:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760704.877048:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.877057:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.877090:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.877183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971437 is committed 00010000:00010000:2.0:1789760704.877224:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16863232->16867327] (req 16863232->16867327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.877325:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507153280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28912us (29231us total) trans 4294971437 rc 0/0 00000100:00100000:2.0:1789760704.877688:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4147 00000100:00100000:2.0:1789760704.877743:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686131891999 time 686163701958 state 0x2 ref 0 00000100:00100000:1.0:1789760704.883452:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760704.883517:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153408 00000100:00100000:2.0:1789760704.883526:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153408 00000100:00100000:2.0:1789760704.883587:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686131891999 time 686163701958 state 0x0 ref 1 00000100:00100000:2.0:1789760704.883598:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686163962374 time 686163701958 state 0x1 ref 1 00000100:00100000:2.0:1789760704.883717:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686163962374 time 686199020563 state 0x0 ref 1 00000100:00100000:2.0:1789760704.883729:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760704.883775:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16867328, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:2.0:1789760704.883782:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.883795:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760704.883799:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.883803:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.883814:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.935487:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971438 is committed 00002000:00010000:3.0:1789760704.935573:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16867328 -> 16871424 00010000:00010000:3.0:1789760704.935588:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935602:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935611:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935618:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760704.935623:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935631:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935651:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.935659:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6cd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16867328->16871423] (req 16867328->16871423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.935833:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52098us (52381us total) trans 4294971438 rc 0/0 00000100:00100000:3.0:1789760704.935844:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4148 00000100:00100000:3.0:1789760704.935853:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686163962374 time 686199020563 state 0x2 ref 0 00000100:00100000:0.0:1789760704.940159:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760704.940671:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153536 00000100:00100000:3.0:1789760704.940681:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153536 00000100:00100000:3.0:1789760704.940762:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686163962374 time 686199020563 state 0x0 ref 1 00000100:00100000:3.0:1789760704.940774:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686199280979 time 686199020563 state 0x1 ref 1 00000100:00100000:0.0:1789760704.940967:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686199280979 time 686256267224 state 0x0 ref 1 00000100:00100000:0.0:1789760704.940983:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507153536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760704.941030:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16871424, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:0.0:1789760704.941038:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.941054:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760704.941059:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.941079:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.941093:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.004136:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16871424 -> 16875520 00010000:00010000:0.0:1789760705.004155:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004168:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004177:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004184:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760705.004189:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004198:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004218:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.004232:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6cdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16871424->16875519] (req 16871424->16875519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.004253:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971439 is committed 00000100:00100000:0.0:1789760705.004325:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507153536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63341us (64171us total) trans 4294971439 rc 0/0 00000100:00100000:0.0:1789760705.004335:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4149 00000100:00100000:0.0:1789760705.004344:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686199280979 time 686256267224 state 0x2 ref 0 00000100:00100000:2.0:1789760705.006836:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760705.006902:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153664 00000100:00100000:1.0:1789760705.006912:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153664 00000100:00100000:1.0:1789760705.006976:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686199280979 time 686256267224 state 0x0 ref 1 00000100:00100000:1.0:1789760705.006990:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686256527640 time 686256267224 state 0x1 ref 1 00000100:00100000:1.0:1789760705.007018:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686256527640 time 686322321105 state 0x0 ref 1 00000100:00100000:1.0:1789760705.007029:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760705.007082:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16875520, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x8807 00010000:00010000:1.0:1789760705.007091:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.007112:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760705.007116:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.007122:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.007136:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760705.007547:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760705.044480:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971440 is committed 00002000:00010000:3.0:1789760705.044518:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16875520 -> 16879616 00002000:00010000:3.0:1789760705.044524:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760703 -> 1789760704 00002000:00010000:3.0:1789760705.044527:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760703 -> 1789760704 00010000:00010000:3.0:1789760705.044541:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044554:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044563:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044571:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.044576:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044587:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044608:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.044616:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6ce3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16875520->16879615] (req 16875520->16879615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.044756:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37723us (37924us total) trans 4294971440 rc 0/0 00000100:00100000:3.0:1789760705.044765:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4150 00000100:00100000:3.0:1789760705.044773:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686256527640 time 686322321105 state 0x2 ref 0 00000100:00100000:1.0:1789760705.053676:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.055354:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153792 00000100:00100000:3.0:1789760705.055369:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153792 00000100:00100000:3.0:1789760705.055437:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686256527640 time 686322321105 state 0x0 ref 1 00000100:00100000:3.0:1789760705.055454:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686322581521 time 686322321105 state 0x1 ref 1 00000100:00100000:3.0:1789760705.055497:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686322581521 time 686370800425 state 0x0 ref 1 00000100:00100000:3.0:1789760705.055510:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.055631:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16879616, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.055641:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.055658:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.055662:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.055668:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.055682:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.084880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971441 is committed 00002000:00010000:3.0:1789760705.085024:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16879616 -> 16883712 00010000:00010000:3.0:1789760705.085040:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085053:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085080:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085087:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.085092:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085101:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085123:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.085130:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16879616->16883711] (req 16879616->16883711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.085219:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29706us (31548us total) trans 4294971441 rc 0/0 00000100:00100000:3.0:1789760705.085229:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4151 00000100:00100000:3.0:1789760705.085239:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686322581521 time 686370800425 state 0x2 ref 0 00000100:00100000:0.0:1789760705.091375:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.091478:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507153920 00000100:00100000:3.0:1789760705.091488:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507153920 00000100:00100000:3.0:1789760705.091651:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686322581521 time 686370800425 state 0x0 ref 1 00000100:00100000:3.0:1789760705.091670:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686371060841 time 686370800425 state 0x1 ref 1 00000100:00100000:3.0:1789760705.091737:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686371060841 time 686407038842 state 0x0 ref 1 00000100:00100000:3.0:1789760705.091751:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.091806:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16883712, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.091830:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.091848:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.091853:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.091860:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.091880:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760705.146317:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16883712 -> 16887808 00010000:00010000:3.0:1789760705.146336:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146348:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146356:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146363:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.146367:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146376:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146406:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.146414:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16883712->16887807] (req 16883712->16887807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.146462:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971442 is committed 00000100:00100000:3.0:1789760705.146518:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507153920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54765us (55148us total) trans 4294971442 rc 0/0 00000100:00100000:3.0:1789760705.146529:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4152 00000100:00100000:3.0:1789760705.146539:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686371060841 time 686407038842 state 0x2 ref 0 00000100:00100000:2.0:1789760705.154869:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.154971:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154048 00000100:00100000:3.0:1789760705.154980:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154048 00000100:00100000:3.0:1789760705.155036:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686371060841 time 686407038842 state 0x0 ref 1 00000100:00100000:3.0:1789760705.155052:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686407299258 time 686407038842 state 0x1 ref 1 00000100:00100000:3.0:1789760705.155091:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686407299258 time 686470393809 state 0x0 ref 1 00000100:00100000:3.0:1789760705.155111:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507154048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.155161:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16887808, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.155170:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.155188:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.155193:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.155200:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.155213:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.190240:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16887808 -> 16891904 00010000:00010000:2.0:1789760705.190266:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190280:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190289:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190304:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.190310:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190318:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190337:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.190345:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6cf8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16887808->16891903] (req 16887808->16891903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.190437:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507154048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35331us (35575us total) trans 4294971443 rc 0/0 00000100:00100000:2.0:1789760705.190447:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4153 00000100:00100000:2.0:1789760705.190457:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686407299258 time 686470393809 state 0x2 ref 0 00000001:00080000:2.0:1789760705.190501:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971443 is committed 00000100:00100000:1.0:1789760705.200466:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.200580:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154176 00000100:00100000:3.0:1789760705.200648:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154176 00000100:00100000:3.0:1789760705.200810:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686407299258 time 686470393809 state 0x0 ref 1 00000100:00100000:3.0:1789760705.200827:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686470654225 time 686470393809 state 0x1 ref 1 00000100:00100000:3.0:1789760705.200856:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686470654225 time 686516159795 state 0x0 ref 1 00000100:00100000:3.0:1789760705.200869:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507154176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.200987:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16891904, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.200996:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.201024:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.201030:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.201037:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.201051:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.228648:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971444 is committed 00002000:00010000:2.0:1789760705.228684:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16891904 -> 16896000 00010000:00010000:2.0:1789760705.228715:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228728:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228740:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228745:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.228749:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228755:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228770:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.228775:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6cff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16891904->16895999] (req 16891904->16895999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.229035:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507154176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28090us (28500us total) trans 4294971444 rc 0/0 00000100:00100000:2.0:1789760705.229048:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4154 00000100:00100000:2.0:1789760705.229068:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686470654225 time 686516159795 state 0x2 ref 0 00000100:00100000:0.0:1789760705.232605:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.232731:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154304 00000100:00100000:3.0:1789760705.232741:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154304 00000100:00100000:3.0:1789760705.232793:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686470654225 time 686516159795 state 0x0 ref 1 00000100:00100000:3.0:1789760705.232809:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686516420211 time 686516159795 state 0x1 ref 1 00000100:00100000:3.0:1789760705.232830:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686516420211 time 686548133867 state 0x0 ref 1 00000100:00100000:3.0:1789760705.232841:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507154304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.232887:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16896000, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.232896:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.232912:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.232916:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.232923:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.232937:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.274270:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16896000 -> 16900096 00010000:00010000:2.0:1789760705.274288:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274305:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274323:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274330:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.274335:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274344:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274364:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.274372:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6d06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16896000->16900095] (req 16896000->16900095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.274470:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507154304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41624us (41870us total) trans 4294971445 rc 0/0 00000100:00100000:2.0:1789760705.274482:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4155 00000100:00100000:2.0:1789760705.274494:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686516420211 time 686548133867 state 0x2 ref 0 00000001:00080000:0.0:1789760705.275291:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971445 is committed 00000100:00100000:0.0:1789760705.279341:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.279833:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154432 00000100:00100000:2.0:1789760705.279845:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154432 00000100:00100000:2.0:1789760705.279902:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686516420211 time 686548133867 state 0x0 ref 1 00000100:00100000:2.0:1789760705.279919:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686548394283 time 686548133867 state 0x1 ref 1 00000100:00100000:2.0:1789760705.279940:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686548394283 time 686595243536 state 0x0 ref 1 00000100:00100000:2.0:1789760705.279952:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507154432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.280001:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16900096, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.280011:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.280026:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760705.280031:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.280037:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.280051:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.323753:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16900096 -> 16904192 00010000:00010000:0.0:1789760705.323775:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323799:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323808:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323816:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760705.323821:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323831:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323850:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.323858:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16900096->16904191] (req 16900096->16904191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.323943:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507154432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43988us (44607us total) trans 4294971446 rc 0/0 00000100:00100000:0.0:1789760705.323958:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4156 00000100:00100000:0.0:1789760705.323968:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686548394283 time 686595243536 state 0x2 ref 0 00000001:00080000:1.0:1789760705.325385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971446 is committed 00000100:00100000:1.0:1789760705.335845:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.336034:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154560 00000100:00100000:2.0:1789760705.336044:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154560 00000100:00100000:2.0:1789760705.336115:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686548394283 time 686595243536 state 0x0 ref 1 00000100:00100000:2.0:1789760705.336130:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686595503952 time 686595243536 state 0x1 ref 1 00000100:00100000:2.0:1789760705.336159:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686595503952 time 686651461909 state 0x0 ref 1 00000100:00100000:2.0:1789760705.336176:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507154560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.336229:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16904192, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.336238:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.336253:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760705.336258:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.336263:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.336277:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.376623:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971447 is committed 00002000:00010000:1.0:1789760705.376847:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16904192 -> 16908288 00010000:00010000:1.0:1789760705.376861:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376874:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376882:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376889:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760705.376893:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376904:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376924:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.376931:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16904192->16908287] (req 16904192->16908287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.377115:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507154560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40936us (41274us total) trans 4294971447 rc 0/0 00000100:00100000:1.0:1789760705.377124:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4157 00000100:00100000:1.0:1789760705.377134:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686595503952 time 686651461909 state 0x2 ref 0 00000100:00100000:2.0:1789760705.383470:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.384438:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154688 00000100:00100000:3.0:1789760705.384448:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154688 00000100:00100000:3.0:1789760705.384496:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686595503952 time 686651461909 state 0x0 ref 1 00000100:00100000:3.0:1789760705.384511:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686651722325 time 686651461909 state 0x1 ref 1 00000100:00100000:3.0:1789760705.384533:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686651722325 time 686699836312 state 0x0 ref 1 00000100:00100000:3.0:1789760705.384544:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507154688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.384594:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16908288, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.384603:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.384618:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.384622:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.384629:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.384642:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.417425:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971448 is committed 00002000:00010000:0.0:1789760705.417531:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16908288 -> 16912384 00010000:00010000:0.0:1789760705.417547:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417561:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417571:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417668:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760705.417673:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417683:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417884:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.417894:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6d1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16908288->16912383] (req 16908288->16912383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.418350:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507154688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33435us (34517us total) trans 4294971448 rc 0/0 00000100:00100000:0.0:1789760705.418362:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4158 00000100:00100000:0.0:1789760705.418371:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686651722325 time 686699836312 state 0x2 ref 0 00000100:00100000:0.0:1789760705.420896:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.421018:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154816 00000100:00100000:2.0:1789760705.421028:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154816 00000100:00100000:2.0:1789760705.421095:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686651722325 time 686699836312 state 0x0 ref 1 00000100:00100000:2.0:1789760705.421110:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686700096728 time 686699836312 state 0x1 ref 1 00000100:00100000:2.0:1789760705.421168:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686700096728 time 686736471014 state 0x0 ref 1 00000100:00100000:2.0:1789760705.421181:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507154816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.421231:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16912384, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.421240:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.421256:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760705.421261:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.421268:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.421283:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.436410:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760705.444780:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971449 is committed 00002000:00010000:3.0:1789760705.444961:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16912384 -> 16916480 00010000:00010000:3.0:1789760705.444976:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.444987:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.445008:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.445014:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.445020:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.445029:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.445048:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.445055:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6d22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16912384->16916479] (req 16912384->16916479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.445277:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507154816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24094us (24384us total) trans 4294971449 rc 0/0 00000100:00100000:3.0:1789760705.445288:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4159 00000100:00100000:3.0:1789760705.445297:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686700096728 time 686736471014 state 0x2 ref 0 00000100:00100000:1.0:1789760705.450789:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.450971:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507154944 00000100:00100000:2.0:1789760705.450985:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507154944 00000100:00100000:2.0:1789760705.451047:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686700096728 time 686736471014 state 0x0 ref 1 00000100:00100000:2.0:1789760705.451071:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686736731430 time 686736471014 state 0x1 ref 1 00000100:00100000:2.0:1789760705.451100:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686736731430 time 686766403718 state 0x0 ref 1 00000100:00100000:2.0:1789760705.451113:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507154944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.451158:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16916480, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.451166:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.451182:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760705.451186:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.451193:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.451212:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.476873:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971450 is committed 00002000:00010000:1.0:1789760705.476959:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16916480 -> 16920576 00010000:00010000:1.0:1789760705.476973:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.476983:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.476989:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.476995:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760705.476998:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.477004:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.477020:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.477024:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b6d29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16916480->16920575] (req 16916480->16920575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.477161:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507154944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26047us (26378us total) trans 4294971450 rc 0/0 00000100:00100000:1.0:1789760705.477169:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4160 00000100:00100000:1.0:1789760705.477176:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686736731430 time 686766403718 state 0x2 ref 0 00000100:00100000:2.0:1789760705.487954:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760705.488208:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155072 00000100:00100000:0.0:1789760705.488218:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155072 00000100:00100000:0.0:1789760705.488271:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686736731430 time 686766403718 state 0x0 ref 1 00000100:00100000:0.0:1789760705.488286:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686766664134 time 686766403718 state 0x1 ref 1 00000100:00100000:0.0:1789760705.488320:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686766664134 time 686803623597 state 0x0 ref 1 00000100:00100000:0.0:1789760705.488334:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507155072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760705.488457:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16920576, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:0.0:1789760705.488466:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.488481:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760705.488486:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.488492:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.488506:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.512753:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971451 is committed 00002000:00010000:0.0:1789760705.512853:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16920576 -> 16924672 00010000:00010000:0.0:1789760705.512868:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512881:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512890:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512897:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760705.512902:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512911:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512932:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.512941:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6d30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16920576->16924671] (req 16920576->16924671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.513031:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507155072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24695us (25083us total) trans 4294971451 rc 0/0 00000100:00100000:0.0:1789760705.513045:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4161 00000100:00100000:0.0:1789760705.513055:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686766664134 time 686803623597 state 0x2 ref 0 00000100:00100000:0.0:1789760705.518490:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.518618:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155200 00000100:00100000:3.0:1789760705.518628:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155200 00000100:00100000:3.0:1789760705.518683:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686766664134 time 686803623597 state 0x0 ref 1 00000100:00100000:3.0:1789760705.518726:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686803884013 time 686803623597 state 0x1 ref 1 00000100:00100000:3.0:1789760705.518750:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686803884013 time 686834053427 state 0x0 ref 1 00000100:00100000:3.0:1789760705.518762:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.518804:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16924672, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.518812:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.518832:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.518836:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.518843:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.518856:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.552113:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971452 is committed 00002000:00010000:3.0:1789760705.552317:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16924672 -> 16928768 00010000:00010000:3.0:1789760705.552333:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552346:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552355:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552362:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.552367:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552376:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552395:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.552403:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16924672->16928767] (req 16924672->16928767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.552499:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33733us (34012us total) trans 4294971452 rc 0/0 00000100:00100000:3.0:1789760705.552585:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4162 00000100:00100000:3.0:1789760705.552595:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686803884013 time 686834053427 state 0x2 ref 0 00000100:00100000:0.0:1789760705.557329:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760705.558250:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155328 00000100:00100000:1.0:1789760705.558261:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155328 00000100:00100000:1.0:1789760705.558316:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686803884013 time 686834053427 state 0x0 ref 1 00000100:00100000:1.0:1789760705.558331:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686834313843 time 686834053427 state 0x1 ref 1 00000100:00100000:1.0:1789760705.558361:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686834313843 time 686873664621 state 0x0 ref 1 00000100:00100000:1.0:1789760705.558374:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760705.558420:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16928768, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:1.0:1789760705.558429:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.558444:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760705.558449:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.558456:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.558469:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760705.589603:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16928768 -> 16932864 00010000:00010000:1.0:1789760705.589626:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589640:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589650:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589657:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760705.589662:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589672:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589715:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.589724:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b6d3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16928768->16932863] (req 16928768->16932863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.589868:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507155328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31490us (32543us total) trans 4294971453 rc 0/0 00000100:00100000:1.0:1789760705.589880:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4163 00000100:00100000:1.0:1789760705.589891:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686834313843 time 686873664621 state 0x2 ref 0 00000001:00080000:1.0:1789760705.590086:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971453 is committed 00000100:00100000:1.0:1789760705.599728:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760705.600080:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155456 00000100:00100000:0.0:1789760705.600222:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155456 00000100:00100000:0.0:1789760705.600283:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686834313843 time 686873664621 state 0x0 ref 1 00000100:00100000:0.0:1789760705.600300:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686873925037 time 686873664621 state 0x1 ref 1 00000100:00100000:0.0:1789760705.600376:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686873925037 time 686915678909 state 0x0 ref 1 00000100:00100000:0.0:1789760705.600389:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507155456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760705.600436:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16932864, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:0.0:1789760705.600445:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.600461:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760705.600466:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.600472:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.600486:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.651147:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16932864 -> 16936960 00010000:00010000:2.0:1789760705.651168:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651181:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651190:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651197:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.651201:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651210:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651229:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.651236:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6d45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16932864->16936959] (req 16932864->16936959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.651333:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507155456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50940us (51608us total) trans 4294971454 rc 0/0 00000100:00100000:2.0:1789760705.651343:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4164 00000100:00100000:2.0:1789760705.651353:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686873925037 time 686915678909 state 0x2 ref 0 00000001:00080000:1.0:1789760705.652199:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971454 is committed 00000100:00100000:0.0:1789760705.655318:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.655402:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155584 00000100:00100000:3.0:1789760705.655411:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155584 00000100:00100000:3.0:1789760705.655478:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686873925037 time 686915678909 state 0x0 ref 1 00000100:00100000:3.0:1789760705.655494:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686915939325 time 686915678909 state 0x1 ref 1 00000100:00100000:3.0:1789760705.655532:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686915939325 time 686970834787 state 0x0 ref 1 00000100:00100000:3.0:1789760705.655545:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.655591:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16936960, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.655601:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.655618:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760705.655622:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.655628:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.655648:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.700862:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971455 is committed 00002000:00010000:3.0:1789760705.701088:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16936960 -> 16941056 00010000:00010000:3.0:1789760705.701110:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701229:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701239:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701246:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760705.701251:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701260:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701283:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.701290:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16936960->16941055] (req 16936960->16941055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.701405:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45856us (46092us total) trans 4294971455 rc 0/0 00000100:00100000:3.0:1789760705.701414:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4165 00000100:00100000:3.0:1789760705.701425:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686915939325 time 686970834787 state 0x2 ref 0 00000100:00100000:1.0:1789760705.706760:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.706948:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155712 00000100:00100000:2.0:1789760705.706958:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155712 00000100:00100000:2.0:1789760705.707014:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686915939325 time 686970834787 state 0x0 ref 1 00000100:00100000:2.0:1789760705.707028:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686971095203 time 686970834787 state 0x1 ref 1 00000100:00100000:2.0:1789760705.707051:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686971095203 time 687022355187 state 0x0 ref 1 00000100:00100000:2.0:1789760705.707073:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507155712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.707128:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16941056, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.707136:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.707151:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760705.707156:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.707162:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.707176:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.748655:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971456 is committed 00002000:00010000:2.0:1789760705.748927:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16941056 -> 16945152 00010000:00010000:2.0:1789760705.748939:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748948:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748953:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748957:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.748961:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748967:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748986:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.748993:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16941056->16945151] (req 16941056->16945151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.749199:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507155712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42121us (42441us total) trans 4294971456 rc 0/0 00000100:00100000:2.0:1789760705.749213:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4166 00000100:00100000:2.0:1789760705.749223:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686971095203 time 687022355187 state 0x2 ref 0 00000100:00100000:1.0:1789760705.752896:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.753051:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155840 00000100:00100000:2.0:1789760705.753070:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155840 00000100:00100000:2.0:1789760705.753130:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 686971095203 time 687022355187 state 0x0 ref 1 00000100:00100000:2.0:1789760705.753144:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687022615603 time 687022355187 state 0x1 ref 1 00000100:00100000:2.0:1789760705.753165:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687022615603 time 687068468863 state 0x0 ref 1 00000100:00100000:2.0:1789760705.753177:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.753229:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16945152, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.753237:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.753262:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760705.753266:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.753272:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.753286:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.876658:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971457 is committed 00002000:00010000:1.0:1789760705.876941:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16945152 -> 16949248 00010000:00010000:1.0:1789760705.876956:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.876969:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.876983:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.876990:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760705.876995:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.877004:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.877023:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.877030:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b6d5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16945152->16949247] (req 16945152->16949247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.877295:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 123966us (124258us total) trans 4294971457 rc 0/0 00000100:00100000:1.0:1789760705.877308:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4167 00000100:00100000:1.0:1789760705.877318:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687022615603 time 687068468863 state 0x2 ref 0 00000100:00100000:0.0:1789760705.884765:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760705.884933:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507155968 00000100:00100000:3.0:1789760705.884943:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507155968 00000100:00100000:3.0:1789760705.884992:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687022615603 time 687068468863 state 0x0 ref 1 00000100:00100000:3.0:1789760705.885007:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687068729279 time 687068468863 state 0x1 ref 1 00000100:00100000:3.0:1789760705.885035:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687068729279 time 687200339171 state 0x0 ref 1 00000100:00100000:3.0:1789760705.885047:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507155968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760705.885107:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16949248, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:3.0:1789760705.885117:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.885132:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760705.885136:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.885142:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.885162:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.929513:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16949248 -> 16953344 00010000:00010000:2.0:1789760705.929533:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929546:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929556:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929564:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760705.929568:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929577:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929597:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.929605:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6d61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16949248->16953343] (req 16949248->16953343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.929730:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507155968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44680us (44969us total) trans 4294971458 rc 0/0 00000100:00100000:2.0:1789760705.929740:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4168 00000100:00100000:2.0:1789760705.929751:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687068729279 time 687200339171 state 0x2 ref 0 00000001:00080000:1.0:1789760705.929841:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971458 is committed 00000100:00100000:0.0:1789760705.940883:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760705.940963:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156096 00000100:00100000:2.0:1789760705.940974:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156096 00000100:00100000:2.0:1789760705.941031:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687068729279 time 687200339171 state 0x0 ref 1 00000100:00100000:2.0:1789760705.941045:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687200599587 time 687200339171 state 0x1 ref 1 00000100:00100000:2.0:1789760705.941084:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687200599587 time 687256387247 state 0x0 ref 1 00000100:00100000:2.0:1789760705.941098:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760705.941145:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16953344, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:2.0:1789760705.941165:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.941181:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760705.941190:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.941196:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.941210:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.031776:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971459 is committed 00002000:00010000:2.0:1789760706.031861:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16953344 -> 16957440 00010000:00010000:2.0:1789760706.031878:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031890:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031900:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031907:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.031911:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031919:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031937:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.031944:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16953344->16957439] (req 16953344->16957439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.032040:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 90939us (91162us total) trans 4294971459 rc 0/0 00000100:00100000:2.0:1789760706.032050:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4169 00000100:00100000:2.0:1789760706.032069:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687200599587 time 687256387247 state 0x2 ref 0 00000100:00100000:0.0:1789760706.036070:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760706.036141:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156224 00000100:00100000:1.0:1789760706.036150:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156224 00000100:00100000:1.0:1789760706.036204:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687200599587 time 687256387247 state 0x0 ref 1 00000100:00100000:1.0:1789760706.036220:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687256647663 time 687256387247 state 0x1 ref 1 00000100:00100000:1.0:1789760706.036252:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687256647663 time 687351555026 state 0x0 ref 1 00000100:00100000:1.0:1789760706.036264:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507156224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760706.036307:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16957440, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x8807 00010000:00010000:1.0:1789760706.036329:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.036350:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760706.036355:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.036361:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.036375:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.111197:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971460 is committed 00002000:00010000:1.0:1789760706.111987:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16957440 -> 16961536 00002000:00010000:1.0:1789760706.111992:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760704 -> 1789760705 00002000:00010000:1.0:1789760706.111995:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760704 -> 1789760705 00010000:00010000:1.0:1789760706.112008:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112021:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112035:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112043:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760706.112047:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112068:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112094:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.112103:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16957440->16961535] (req 16957440->16961535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.112197:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507156224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75930us (76150us total) trans 4294971460 rc 0/0 00000100:00100000:1.0:1789760706.112207:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4170 00000100:00100000:1.0:1789760706.112217:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687256647663 time 687351555026 state 0x2 ref 0 00000100:00100000:1.0:1789760706.127884:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760706.128046:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156352 00000100:00100000:2.0:1789760706.128066:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156352 00000100:00100000:2.0:1789760706.128125:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687256647663 time 687351555026 state 0x0 ref 1 00000100:00100000:2.0:1789760706.128141:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687351815442 time 687351555026 state 0x1 ref 1 00000100:00100000:2.0:1789760706.128162:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687351815442 time 687443466274 state 0x0 ref 1 00000100:00100000:2.0:1789760706.128175:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507156352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760706.128226:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16961536, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:2.0:1789760706.128235:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.128252:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760706.128256:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.128263:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.128278:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.225039:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971461 is committed 00002000:00010000:2.0:1789760706.225335:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16961536 -> 16965632 00010000:00010000:2.0:1789760706.225351:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225362:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225372:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225378:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.225383:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225392:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225411:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.225419:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16961536->16965631] (req 16961536->16965631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.225525:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507156352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 97347us (97646us total) trans 4294971461 rc 0/0 00000100:00100000:2.0:1789760706.225535:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4171 00000100:00100000:2.0:1789760706.225544:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687351815442 time 687443466274 state 0x2 ref 0 00000100:00100000:0.0:1789760706.243867:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760706.245769:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156480 00000100:00100000:2.0:1789760706.245783:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156480 00000100:00100000:2.0:1789760706.245857:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687351815442 time 687443466274 state 0x0 ref 1 00000100:00100000:2.0:1789760706.245873:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687443726690 time 687443466274 state 0x1 ref 1 00000100:00100000:2.0:1789760706.245896:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687443726690 time 687561199341 state 0x0 ref 1 00000100:00100000:2.0:1789760706.245909:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760706.245956:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16965632, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:2.0:1789760706.245965:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.245982:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760706.245986:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.245993:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.246009:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.410984:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971462 is committed 00002000:00010000:2.0:1789760706.411101:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16965632 -> 16969728 00010000:00010000:2.0:1789760706.411117:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411130:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411140:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411258:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.411263:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411274:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411300:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.411309:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b6d7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16965632->16969727] (req 16965632->16969727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.411518:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 165607us (167655us total) trans 4294971462 rc 0/0 00000100:00100000:2.0:1789760706.411529:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4172 00000100:00100000:2.0:1789760706.411539:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687443726690 time 687561199341 state 0x2 ref 0 00000100:00100000:3.0:1789760706.420354:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760706.420549:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156608 00000100:00100000:1.0:1789760706.420559:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156608 00000100:00100000:1.0:1789760706.420612:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687443726690 time 687561199341 state 0x0 ref 1 00000100:00100000:1.0:1789760706.420631:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687561459757 time 687561199341 state 0x1 ref 1 00000100:00100000:1.0:1789760706.420664:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687561459757 time 687735967128 state 0x0 ref 1 00000100:00100000:1.0:1789760706.420677:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760706.420771:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16969728, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:1.0:1789760706.420781:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.420796:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760706.420802:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.420808:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.420822:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.476984:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760706.539167:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971463 is committed 00002000:00010000:3.0:1789760706.540804:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16969728 -> 16973824 00010000:00010000:3.0:1789760706.540835:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540850:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540860:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540868:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760706.540874:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540884:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540903:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.540911:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6d84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16969728->16973823] (req 16969728->16973823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.541022:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 120340us (120678us total) trans 4294971463 rc 0/0 00000100:00100000:3.0:1789760706.541032:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4173 00000100:00100000:3.0:1789760706.541042:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687561459757 time 687735967128 state 0x2 ref 0 00000100:00100000:0.0:1789760706.549150:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760706.550741:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156736 00000100:00100000:3.0:1789760706.550755:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156736 00000100:00100000:3.0:1789760706.550818:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687561459757 time 687735967128 state 0x0 ref 1 00000100:00100000:3.0:1789760706.550845:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687736227544 time 687735967128 state 0x1 ref 1 00000100:00100000:3.0:1789760706.550878:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687736227544 time 687866181592 state 0x0 ref 1 00000100:00100000:3.0:1789760706.550892:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760706.550943:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16973824, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:3.0:1789760706.550953:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.550969:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760706.550974:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.550981:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.550994:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.621065:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971464 is committed 00002000:00010000:3.0:1789760706.621307:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16973824 -> 16977920 00010000:00010000:3.0:1789760706.621323:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621336:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621346:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621353:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760706.621358:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621368:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621385:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.621394:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6d8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16973824->16977919] (req 16973824->16977919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.621482:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70588us (72348us total) trans 4294971464 rc 0/0 00000100:00100000:3.0:1789760706.621504:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4174 00000100:00100000:3.0:1789760706.621514:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687736227544 time 687866181592 state 0x2 ref 0 00000100:00100000:0.0:1789760706.636066:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760706.636674:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156864 00000100:00100000:2.0:1789760706.636685:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156864 00000100:00100000:2.0:1789760706.636779:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687736227544 time 687866181592 state 0x0 ref 1 00000100:00100000:2.0:1789760706.636795:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687866442008 time 687866181592 state 0x1 ref 1 00000100:00100000:2.0:1789760706.636816:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687866442008 time 687952119788 state 0x0 ref 1 00000100:00100000:2.0:1789760706.636829:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760706.636877:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16977920, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:2.0:1789760706.636887:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.636909:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760706.636913:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.636920:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.636934:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.714847:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971465 is committed 00002000:00010000:2.0:1789760706.714960:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16977920 -> 16982016 00010000:00010000:2.0:1789760706.714976:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.714989:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.714997:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.715005:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.715009:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.715019:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.715041:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.715062:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6d92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16977920->16982015] (req 16977920->16982015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.715156:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78324us (79106us total) trans 4294971465 rc 0/0 00000100:00100000:2.0:1789760706.715166:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4175 00000100:00100000:2.0:1789760706.715175:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687866442008 time 687952119788 state 0x2 ref 0 00000100:00100000:2.0:1789760706.725290:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760706.725828:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507156992 00000100:00100000:3.0:1789760706.725839:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507156992 00000100:00100000:3.0:1789760706.725896:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687866442008 time 687952119788 state 0x0 ref 1 00000100:00100000:3.0:1789760706.725910:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687952380204 time 687952119788 state 0x1 ref 1 00000100:00100000:3.0:1789760706.725936:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687952380204 time 688041239319 state 0x0 ref 1 00000100:00100000:3.0:1789760706.725948:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760706.725995:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16982016, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:3.0:1789760706.726005:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.726022:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760706.726027:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.726034:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.726066:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.818365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971466 is committed 00002000:00010000:2.0:1789760706.825819:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16982016 -> 16986112 00010000:00010000:2.0:1789760706.825841:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825856:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825865:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825874:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.825878:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825888:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825915:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.825923:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6d99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16982016->16986111] (req 16982016->16986111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.826432:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507156992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 100481us (101146us total) trans 4294971466 rc 0/0 00000100:00100000:2.0:1789760706.826443:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4176 00000100:00100000:2.0:1789760706.826452:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687952380204 time 688041239319 state 0x2 ref 0 00000100:00100000:3.0:1789760706.843059:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760706.843198:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157120 00000100:00100000:2.0:1789760706.843208:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157120 00000100:00100000:2.0:1789760706.843263:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 687952380204 time 688041239319 state 0x0 ref 1 00000100:00100000:2.0:1789760706.843278:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688041499735 time 688041239319 state 0x1 ref 1 00000100:00100000:2.0:1789760706.843308:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688041499735 time 688158612222 state 0x0 ref 1 00000100:00100000:2.0:1789760706.843321:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507157120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760706.843369:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16986112, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:2.0:1789760706.843378:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.843393:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760706.843397:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.843403:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.843415:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.920406:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971467 is committed 00002000:00010000:2.0:1789760706.921750:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16986112 -> 16990208 00010000:00010000:2.0:1789760706.921767:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921780:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921789:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921795:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760706.921800:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921808:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921828:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.921841:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6da0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16986112->16990207] (req 16986112->16990207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.922078:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507157120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78753us (79030us total) trans 4294971467 rc 0/0 00000100:00100000:2.0:1789760706.922100:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4177 00000100:00100000:2.0:1789760706.922110:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688041499735 time 688158612222 state 0x2 ref 0 00000100:00100000:3.0:1789760706.929828:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760706.931344:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157248 00000100:00100000:2.0:1789760706.931356:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157248 00000100:00100000:2.0:1789760706.931414:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688041499735 time 688158612222 state 0x0 ref 1 00000100:00100000:2.0:1789760706.931428:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688158872638 time 688158612222 state 0x1 ref 1 00000100:00100000:2.0:1789760706.931462:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688158872638 time 688246765239 state 0x0 ref 1 00000100:00100000:2.0:1789760706.931475:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507157248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760706.931761:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16990208, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:1.0:1789760706.931772:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.931788:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760706.931792:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.931798:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.931817:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.980826:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971468 is committed 00002000:00010000:0.0:1789760706.983591:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16990208 -> 16994304 00010000:00010000:0.0:1789760706.983677:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983705:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983715:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983722:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760706.983727:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983735:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983755:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.983762:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b6da7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16990208->16994303] (req 16990208->16994303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.983918:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507157248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52438us (54093us total) trans 4294971468 rc 0/0 00000100:00100000:0.0:1789760706.983928:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4178 00000100:00100000:0.0:1789760706.983937:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688158872638 time 688246765239 state 0x2 ref 0 00000100:00100000:3.0:1789760706.991306:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760706.997255:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157376 00000100:00100000:0.0:1789760706.997270:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157376 00000100:00100000:1.0:1789760706.999060:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688158872638 time 688246765239 state 0x0 ref 1 00000100:00100000:1.0:1789760706.999078:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688247025655 time 688246765239 state 0x1 ref 1 00000100:00100000:1.0:1789760706.999125:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688247025655 time 688314429212 state 0x0 ref 1 00000100:00100000:1.0:1789760706.999139:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507157376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760706.999189:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16994304, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x8807 00010000:00010000:1.0:1789760706.999198:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.999215:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760706.999219:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.999226:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.999239:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.097860:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971469 is committed 00002000:00010000:1.0:1789760707.098669:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16994304 -> 16998400 00002000:00010000:1.0:1789760707.098675:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760705 -> 1789760706 00002000:00010000:1.0:1789760707.098677:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760705 -> 1789760706 00010000:00010000:1.0:1789760707.098720:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098734:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098745:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098752:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760707.098757:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098766:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098787:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.098795:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6dae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16994304->16998399] (req 16994304->16998399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.098892:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507157376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 99749us (107590us total) trans 4294971469 rc 0/0 00000100:00100000:1.0:1789760707.098902:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4179 00000100:00100000:1.0:1789760707.098911:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688247025655 time 688314429212 state 0x2 ref 0 00000100:00100000:2.0:1789760707.102810:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760707.103006:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157504 00000100:00100000:3.0:1789760707.103016:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157504 00000100:00080000:0.0:1789760707.103295:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760707.103375:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688247025655 time 688314429212 state 0x0 ref 1 00000100:00100000:3.0:1789760707.103392:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688314689628 time 688314429212 state 0x1 ref 1 00000100:00100000:3.0:1789760707.103421:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688314689628 time 688418724076 state 0x0 ref 1 00000100:00100000:3.0:1789760707.103434:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507157504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760707.103482:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16998400, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:3.0:1789760707.103594:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.103675:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760707.103680:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.103687:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.103774:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.232841:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971470 is committed 00002000:00010000:3.0:1789760707.233037:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 16998400 -> 17002496 00010000:00010000:3.0:1789760707.233070:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233231:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233242:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233249:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760707.233254:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233264:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233283:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.233292:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6db5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16998400->17002495] (req 16998400->17002495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.233390:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507157504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 129953us (130583us total) trans 4294971470 rc 0/0 00000100:00100000:3.0:1789760707.233401:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4180 00000100:00100000:3.0:1789760707.233410:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688314689628 time 688418724076 state 0x2 ref 0 00000100:00100000:1.0:1789760707.240246:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.240594:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157632 00000100:00100000:0.0:1789760707.240604:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157632 00000100:00100000:0.0:1789760707.240660:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688314689628 time 688418724076 state 0x0 ref 1 00000100:00100000:0.0:1789760707.240676:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688418984492 time 688418724076 state 0x1 ref 1 00000100:00100000:0.0:1789760707.240732:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688418984492 time 688556034720 state 0x0 ref 1 00000100:00100000:0.0:1789760707.240745:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507157632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760707.240793:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17002496, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:0.0:1789760707.240801:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.240818:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760707.240823:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.240829:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.240843:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.289499:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.289533:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.289557:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.289578:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.289650:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.289681:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157888 00000100:00100000:0.0:1789760707.289689:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157888 00000100:00100000:0.0:1789760707.289762:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158144 00000100:00100000:0.0:1789760707.289765:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158144 00000100:00100000:0.0:1789760707.289779:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158016 00000100:00100000:0.0:1789760707.289781:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158016 00000100:00100000:0.0:1789760707.289793:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158272 00000100:00100000:0.0:1789760707.289795:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158272 00000100:00100000:0.0:1789760707.289810:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 818 00000100:00100000:0.0:1789760707.289831:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507157888:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760707.289903:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507157888:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 84us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.289910:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 818 00000100:00100000:0.0:1789760707.289926:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 819 00000100:00100000:0.0:1789760707.289933:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507158144:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760707.289957:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507158144:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.289964:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 819 00000100:00100000:0.0:1789760707.289974:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 820 00000100:00100000:0.0:1789760707.289980:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158016:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760707.290002:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158016:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.290008:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 820 00000100:00100000:0.0:1789760707.290018:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 821 00000100:00100000:0.0:1789760707.290023:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507158272:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760707.290055:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507158272:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 21us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.290061:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 821 00000100:00100000:0.0:1789760707.290089:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507157760 00000100:00100000:0.0:1789760707.290093:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507157760 00000100:00100000:0.0:1789760707.290118:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 454 00000100:00100000:0.0:1789760707.290124:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507157760:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760707.290148:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507157760:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (590us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.290154:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 454 00000100:00080000:0.0:1789760707.290183:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760707.290187:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760707.310670:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971471 is committed 00002000:00010000:2.0:1789760707.310949:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17002496 -> 17006592 00010000:00010000:2.0:1789760707.310966:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.310978:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.310989:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.310997:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.311001:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.311011:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.311031:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.311039:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6dbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17002496->17006591] (req 17002496->17006591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.311157:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507157632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70408us (70916us total) trans 4294971471 rc 0/0 00000100:00100000:2.0:1789760707.311166:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4181 00000100:00100000:2.0:1789760707.311175:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688418984492 time 688556034720 state 0x2 ref 0 00000100:00100000:0.0:1789760707.323763:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760707.323858:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158400 00000100:00100000:2.0:1789760707.323869:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158400 00000100:00100000:2.0:1789760707.323923:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688418984492 time 688556034720 state 0x0 ref 1 00000100:00100000:2.0:1789760707.323938:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688556295136 time 688556034720 state 0x1 ref 1 00000100:00100000:2.0:1789760707.323969:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688556295136 time 688639272428 state 0x0 ref 1 00000100:00100000:2.0:1789760707.323981:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760707.324031:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17006592, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760707.324040:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.324078:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760707.324083:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.324089:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.324112:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.363436:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971472 is committed 00002000:00010000:2.0:1789760707.363543:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17006592 -> 17010688 00010000:00010000:2.0:1789760707.363560:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363573:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363582:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363589:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.363593:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363602:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363620:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.363626:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17006592->17010687] (req 17006592->17010687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.363749:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39764us (39991us total) trans 4294971472 rc 0/0 00000100:00100000:2.0:1789760707.363759:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4182 00000100:00100000:2.0:1789760707.363770:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688556295136 time 688639272428 state 0x2 ref 0 00000100:00100000:1.0:1789760707.369970:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760707.370388:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158528 00000100:00100000:3.0:1789760707.370398:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158528 00000100:00100000:3.0:1789760707.370450:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688556295136 time 688639272428 state 0x0 ref 1 00000100:00100000:3.0:1789760707.370464:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688639532844 time 688639272428 state 0x1 ref 1 00000100:00100000:3.0:1789760707.370489:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688639532844 time 688685793206 state 0x0 ref 1 00000100:00100000:3.0:1789760707.370511:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760707.370555:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17010688, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:3.0:1789760707.370571:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.370587:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760707.370591:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.370597:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.370611:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.417801:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971473 is committed 00002000:00010000:2.0:1789760707.417909:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17010688 -> 17014784 00010000:00010000:2.0:1789760707.417925:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.417938:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.417948:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.417960:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.417965:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.417975:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.417995:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.418002:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6dca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17010688->17014783] (req 17010688->17014783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.418297:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47780us (48329us total) trans 4294971473 rc 0/0 00000100:00100000:2.0:1789760707.418307:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4183 00000100:00100000:2.0:1789760707.418317:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688639532844 time 688685793206 state 0x2 ref 0 00000100:00100000:2.0:1789760707.442766:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.442849:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158656 00000100:00100000:0.0:1789760707.442860:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158656 00000100:00100000:0.0:1789760707.442917:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688639532844 time 688685793206 state 0x0 ref 1 00000100:00100000:0.0:1789760707.442933:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688686053622 time 688685793206 state 0x1 ref 1 00000100:00100000:0.0:1789760707.442968:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688686053622 time 688758271459 state 0x0 ref 1 00000100:00100000:0.0:1789760707.442980:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760707.443027:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17014784, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:0.0:1789760707.443036:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.443071:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760707.443077:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.443089:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.443105:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.484887:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760707.487945:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17014784 -> 17018880 00010000:00010000:0.0:1789760707.487963:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.487983:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.487991:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.487999:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760707.488003:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.488012:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.488032:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.488040:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6dd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17014784->17018879] (req 17014784->17018879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.488156:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507158656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45173us (45395us total) trans 4294971474 rc 0/0 00000100:00100000:0.0:1789760707.488166:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4184 00000100:00100000:0.0:1789760707.488176:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688686053622 time 688758271459 state 0x2 ref 0 00000001:00080000:1.0:1789760707.488736:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971474 is committed 00000100:00100000:0.0:1789760707.499890:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760707.500536:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158784 00000100:00100000:2.0:1789760707.500545:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158784 00000100:00100000:2.0:1789760707.500598:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688686053622 time 688758271459 state 0x0 ref 1 00000100:00100000:2.0:1789760707.500615:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688758531875 time 688758271459 state 0x1 ref 1 00000100:00100000:2.0:1789760707.500647:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688758531875 time 688815949855 state 0x0 ref 1 00000100:00100000:2.0:1789760707.500660:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507158784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760707.500742:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17018880, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760707.500751:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.500766:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760707.500771:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.500777:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.500790:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.631579:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971475 is committed 00002000:00010000:0.0:1789760707.631785:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17018880 -> 17022976 00010000:00010000:0.0:1789760707.631801:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631814:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631824:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631831:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760707.631836:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631845:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631865:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.631873:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6dd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17018880->17022975] (req 17018880->17022975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.631966:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507158784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 131302us (132079us total) trans 4294971475 rc 0/0 00000100:00100000:0.0:1789760707.631977:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4185 00000100:00100000:0.0:1789760707.631986:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688758531875 time 688815949855 state 0x2 ref 0 00000100:00100000:3.0:1789760707.635649:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760707.635774:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507158912 00000100:00100000:2.0:1789760707.635784:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507158912 00000100:00100000:2.0:1789760707.635842:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688758531875 time 688815949855 state 0x0 ref 1 00000100:00100000:2.0:1789760707.635859:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688816210271 time 688815949855 state 0x1 ref 1 00000100:00100000:2.0:1789760707.635893:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688816210271 time 688951196029 state 0x0 ref 1 00000100:00100000:2.0:1789760707.635907:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507158912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760707.635955:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17022976, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760707.635964:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.635986:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760707.635991:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.635997:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.636011:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.713742:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17022976 -> 17027072 00010000:00010000:2.0:1789760707.713772:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713786:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713795:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713802:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.713808:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713817:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713839:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.713848:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6ddf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17022976->17027071] (req 17022976->17027071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.713932:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507158912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78023us (78289us total) trans 4294971476 rc 0/0 00000100:00100000:2.0:1789760707.713943:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4186 00000100:00100000:2.0:1789760707.713953:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688816210271 time 688951196029 state 0x2 ref 0 00000001:00080000:1.0:1789760707.714852:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971476 is committed 00000100:00100000:2.0:1789760707.718326:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760707.718469:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159040 00000100:00100000:0.0:1789760707.718479:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159040 00000100:00100000:0.0:1789760707.718556:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688816210271 time 688951196029 state 0x0 ref 1 00000100:00100000:0.0:1789760707.718570:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688951456445 time 688951196029 state 0x1 ref 1 00000100:00100000:0.0:1789760707.718600:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688951456445 time 689033903844 state 0x0 ref 1 00000100:00100000:0.0:1789760707.718612:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240b8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760707.718660:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17027072, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:0.0:1789760707.718669:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.718684:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760707.718688:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.718730:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.718744:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.804057:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104069c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538257280:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760707.804368:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538257664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.804413:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804438:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538257408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.804470:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538258176:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804471:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804494:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538257920:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804517:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804535:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538258432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804554:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804569:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538258944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804589:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804612:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789760707.804618:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538257536:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804641:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804659:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538258048:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804679:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804709:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538258560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804732:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760707.804759:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257408 00000100:00100000:2.0:1789760707.804766:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257408 00000100:00100000:2.0:1789760707.804808:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760707.804817:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2234cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538257408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.804867:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2234cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538257408:12345-0@lo:400:kworker.0 Request processed in 51us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760707.804874:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760707.804903:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257536 00000100:00100000:2.0:1789760707.804907:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257536 00000100:00100000:2.0:1789760707.804930:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258048 00000100:00100000:2.0:1789760707.804933:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258048 00000100:00100000:2.0:1789760707.804976:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258560 00000100:00100000:2.0:1789760707.804980:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258560 00000100:00100000:2.0:1789760707.804996:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 822 00000100:00100000:2.0:1789760707.805002:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538257536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805035:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538257536:12345-0@lo:400:kworker.0 Request processed in 33us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760707.805052:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 822 00000100:00100000:2.0:1789760707.805155:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 823 00000100:00100000:2.0:1789760707.805162:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538258048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805193:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538258048:12345-0@lo:400:kworker.0 Request processed in 32us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760707.805199:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 823 00000100:00100000:2.0:1789760707.805211:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 824 00000100:00100000:2.0:1789760707.805218:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538258560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805245:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538258560:12345-0@lo:400:kworker.0 Request processed in 28us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760707.805251:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 824 00000100:00100000:2.0:1789760707.805297:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538257536:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805307:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538258048:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805315:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538258560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760707.805322:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538257408:0@lo:400:kworker.0 00000100:00080000:2.0:1789760707.805351:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760707.805438:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257664 00000100:00100000:0.0:1789760707.805442:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257664 00000100:00100000:0.0:1789760707.805475:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257920 00000100:00100000:0.0:1789760707.805479:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257920 00000100:00100000:0.0:1789760707.805494:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760707.805500:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538257664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805538:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538257664:12345-0@lo:400:kworker.0 Request processed in 38us (1125us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.805544:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760707.805555:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789760707.805561:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538257920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805591:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538257920:12345-0@lo:400:kworker.0 Request processed in 31us (1073us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.805597:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789760707.805621:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538257920:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805763:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.805792:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538258688:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805799:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258176 00000100:00100000:0.0:1789760707.805803:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258176 00000100:00100000:3.0:1789760707.805818:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760707.805832:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258688 00000100:00100000:0.0:1789760707.805835:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258688 00000100:00100000:3.0:1789760707.805842:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538257664:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805852:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 825 00000100:00100000:3.0:1789760707.805853:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1789760707.805858:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538258176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805858:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538257792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805879:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760707.805885:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538258176:12345-0@lo:400:kworker.0 Request processed in 28us (122us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.805891:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 825 00000100:00100000:3.0:1789760707.805895:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538258304:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805901:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257792 00000100:00100000:0.0:1789760707.805903:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257792 00000100:00100000:0.0:1789760707.805916:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 826 00000100:00100000:3.0:1789760707.805916:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760707.805921:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538258688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805933:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538258816:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805946:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538258688:12345-0@lo:400:kworker.0 Request processed in 25us (127us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.805951:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760707.805953:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 826 00000100:00100000:0.0:1789760707.805964:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 827 00000100:00100000:0.0:1789760707.805970:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538257792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805984:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538258688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.805995:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538258176:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.805996:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538257792:12345-0@lo:400:kworker.0 Request processed in 27us (116us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760707.806001:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 827 00000100:00100000:3.0:1789760707.806009:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538257792:0@lo:400:kworker.0 00000100:00080000:0.0:1789760707.806019:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760707.806029:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258432 00000100:00100000:3.0:1789760707.806032:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258432 00000100:00100000:0.0:1789760707.806035:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.806084:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258944 00000100:00100000:3.0:1789760707.806088:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258944 00000100:00100000:1.0:1789760707.806248:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538259072:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760707.806276:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760707.806304:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538257280 00000100:00100000:1.0:1789760707.806307:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538257280 00000100:00100000:1.0:1789760707.806331:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259072 00000100:00100000:1.0:1789760707.806333:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259072 00000100:00100000:1.0:1789760707.806345:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789760707.806351:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538257280:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760707.806378:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760707.806412:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538257280:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760707.806419:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789760707.806431:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1789760707.806437:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3464:x1876699538259072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760707.806448:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760707.806473:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3464:x1876699538259072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760707.806478:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1789760707.806508:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538259072:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760707.806536:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104069c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538257280:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760707.807751:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258304 00000100:00100000:3.0:1789760707.807762:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258304 00000100:00100000:3.0:1789760707.807785:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538258816 00000100:00100000:3.0:1789760707.807788:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538258816 00000100:00100000:3.0:1789760707.807805:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789760707.807813:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538258432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.807885:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538258432:12345-0@lo:400:kworker.0 Request processed in 72us (3328us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.807892:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789760707.807906:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789760707.807912:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538258944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.807939:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538258944:12345-0@lo:400:kworker.0 Request processed in 28us (3349us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.807945:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789760707.807957:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789760707.807963:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538258304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.808013:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538258304:12345-0@lo:400:kworker.0 Request processed in 52us (2097us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.808019:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789760707.808029:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789760707.808035:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538258816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.808072:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538258816:12345-0@lo:400:kworker.0 Request processed in 37us (2119us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.808077:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1789760707.808183:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538258304:0@lo:400:kworker.0 00000100:00100000:1.0:1789760707.808194:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538258816:0@lo:400:kworker.0 00000100:00080000:3.0:1789760707.809277:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760707.809284:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760707.809445:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971477 is committed 00002000:00010000:2.0:1789760707.809474:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17027072 -> 17031168 00010000:00010000:2.0:1789760707.809489:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809500:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809508:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809515:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.809519:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809528:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809548:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.809556:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6de6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17027072->17031167] (req 17027072->17031167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.809630:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240b8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91016us (91309us total) trans 4294971477 rc 0/0 00000100:00100000:2.0:1789760707.809640:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4187 00000100:00100000:2.0:1789760707.809649:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688951456445 time 689033903844 state 0x2 ref 0 00000100:00100000:0.0:1789760707.809935:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538258432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.809949:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538258944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760707.821636:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760707.821726:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159168 00000100:00100000:3.0:1789760707.821737:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159168 00000100:00100000:3.0:1789760707.821791:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 688951456445 time 689033903844 state 0x0 ref 1 00000100:00100000:3.0:1789760707.821805:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689034164260 time 689033903844 state 0x1 ref 1 00000100:00100000:3.0:1789760707.821828:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689034164260 time 689137130702 state 0x0 ref 1 00000100:00100000:3.0:1789760707.821840:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507159168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760707.821889:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17031168, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:3.0:1789760707.821898:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.821914:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760707.821918:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.821925:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.821939:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.880003:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971478 is committed 00002000:00010000:3.0:1789760707.880259:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17031168 -> 17035264 00010000:00010000:3.0:1789760707.880274:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880287:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880296:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880303:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760707.880309:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880318:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880336:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.880345:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6ded lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17031168->17035263] (req 17031168->17035263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.880510:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507159168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58667us (58878us total) trans 4294971478 rc 0/0 00000100:00100000:3.0:1789760707.880520:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4188 00000100:00100000:3.0:1789760707.880529:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689034164260 time 689137130702 state 0x2 ref 0 00000100:00100000:3.0:1789760707.883908:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760707.884325:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159296 00000100:00100000:2.0:1789760707.884337:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159296 00000100:00100000:2.0:1789760707.884397:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689034164260 time 689137130702 state 0x0 ref 1 00000100:00100000:2.0:1789760707.884412:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689137391118 time 689137130702 state 0x1 ref 1 00000100:00100000:2.0:1789760707.884440:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689137391118 time 689199743755 state 0x0 ref 1 00000100:00100000:2.0:1789760707.884451:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760707.884554:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17035264, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760707.884564:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.884786:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760707.884794:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.884801:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.884821:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.929600:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971479 is committed 00002000:00010000:2.0:1789760707.929866:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17035264 -> 17039360 00010000:00010000:2.0:1789760707.929882:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929894:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929904:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929911:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760707.929915:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929923:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929944:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.929952:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6df4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17035264->17039359] (req 17035264->17039359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.930059:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45604us (46155us total) trans 4294971479 rc 0/0 00000100:00100000:2.0:1789760707.930069:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4189 00000100:00100000:2.0:1789760707.930078:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689137391118 time 689199743755 state 0x2 ref 0 00000100:00100000:0.0:1789760707.933928:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760707.934022:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159424 00000100:00100000:2.0:1789760707.934032:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159424 00000100:00100000:2.0:1789760707.934208:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689137391118 time 689199743755 state 0x0 ref 1 00000100:00100000:2.0:1789760707.934224:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689200004171 time 689199743755 state 0x1 ref 1 00000100:00100000:2.0:1789760707.934281:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689200004171 time 689249584722 state 0x0 ref 1 00000100:00100000:2.0:1789760707.934293:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760707.934341:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17039360, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760707.934349:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.934365:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760707.934370:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.934376:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.934398:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.972390:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971480 is committed 00002000:00010000:3.0:1789760707.972982:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17039360 -> 17043456 00010000:00010000:3.0:1789760707.972999:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973012:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973021:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973034:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760707.973049:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973058:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973077:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.973181:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6dfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17039360->17043455] (req 17039360->17043455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.973308:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39011us (39385us total) trans 4294971480 rc 0/0 00000100:00100000:3.0:1789760707.973318:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4190 00000100:00100000:3.0:1789760707.973327:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689200004171 time 689249584722 state 0x2 ref 0 00000100:00100000:0.0:1789760707.978534:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760707.978744:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159552 00000100:00100000:3.0:1789760707.978754:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159552 00000100:00100000:3.0:1789760707.978804:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689200004171 time 689249584722 state 0x0 ref 1 00000100:00100000:3.0:1789760707.978818:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689249845138 time 689249584722 state 0x1 ref 1 00000100:00100000:3.0:1789760707.978838:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689249845138 time 689294142517 state 0x0 ref 1 00000100:00100000:3.0:1789760707.978852:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507159552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760707.978896:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17043456, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:3.0:1789760707.978904:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.978919:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760707.978923:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.978929:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.978943:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.036499:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971481 is committed 00002000:00010000:1.0:1789760708.037742:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17043456 -> 17047552 00010000:00010000:1.0:1789760708.037760:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037774:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037783:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037790:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760708.037795:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037803:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037834:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.037843:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6e02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17043456->17047551] (req 17043456->17047551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.037948:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507159552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59092us (59420us total) trans 4294971481 rc 0/0 00000100:00100000:1.0:1789760708.037959:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4191 00000100:00100000:1.0:1789760708.039311:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689249845138 time 689294142517 state 0x2 ref 0 00000100:00100000:1.0:1789760708.042020:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.042222:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159680 00000100:00100000:2.0:1789760708.042232:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159680 00000100:00100000:2.0:1789760708.042290:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689249845138 time 689294142517 state 0x0 ref 1 00000100:00100000:2.0:1789760708.042305:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689294402933 time 689294142517 state 0x1 ref 1 00000100:00100000:2.0:1789760708.042336:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689294402933 time 689357639441 state 0x0 ref 1 00000100:00100000:2.0:1789760708.042353:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.042397:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17047552, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x8807 00010000:00010000:2.0:1789760708.042406:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.042420:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760708.042424:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.042430:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.042444:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.170463:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971482 is committed 00002000:00010000:2.0:1789760708.172589:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17047552 -> 17051648 00002000:00010000:2.0:1789760708.172597:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:2.0:1789760708.172600:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:2.0:1789760708.172615:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172628:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172638:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172644:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.172649:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172658:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172679:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.172686:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17047552->17051647] (req 17047552->17051647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.173812:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507159680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 131452us (131792us total) trans 4294971482 rc 0/0 00000100:00100000:2.0:1789760708.173839:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4192 00000100:00100000:2.0:1789760708.173850:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689294402933 time 689357639441 state 0x2 ref 0 00000100:00100000:2.0:1789760708.187536:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760708.187622:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159808 00000100:00100000:3.0:1789760708.187632:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159808 00000100:00100000:3.0:1789760708.187688:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689294402933 time 689357639441 state 0x0 ref 1 00000100:00100000:3.0:1789760708.187755:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689357899857 time 689357639441 state 0x1 ref 1 00000100:00100000:3.0:1789760708.187785:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689357899857 time 689503088367 state 0x0 ref 1 00000100:00100000:3.0:1789760708.187799:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760708.187863:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17051648, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:3.0:1789760708.187873:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.187890:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760708.187894:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.187901:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.187916:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.304457:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971483 is committed 00002000:00010000:2.0:1789760708.304871:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17051648 -> 17055744 00010000:00010000:2.0:1789760708.304888:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304900:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304910:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304917:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.304922:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304931:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304950:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.304958:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6e10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17051648->17055743] (req 17051648->17055743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.305059:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 117258us (117529us total) trans 4294971483 rc 0/0 00000100:00100000:2.0:1789760708.305082:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4193 00000100:00100000:2.0:1789760708.305098:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689357899857 time 689503088367 state 0x2 ref 0 00000100:00100000:2.0:1789760708.313590:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760708.313940:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507159936 00000100:00100000:1.0:1789760708.313949:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507159936 00000100:00100000:1.0:1789760708.314003:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689357899857 time 689503088367 state 0x0 ref 1 00000100:00100000:1.0:1789760708.314018:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689503348783 time 689503088367 state 0x1 ref 1 00000100:00100000:1.0:1789760708.314066:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689503348783 time 689629360734 state 0x0 ref 1 00000100:00100000:1.0:1789760708.314092:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760708.314301:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17055744, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:1.0:1789760708.314311:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.314328:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760708.314332:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.314339:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.314354:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.409272:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971484 is committed 00002000:00010000:2.0:1789760708.415238:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17055744 -> 17059840 00010000:00010000:2.0:1789760708.415259:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415274:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415284:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415291:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.415296:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415305:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415326:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.415333:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6e17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17055744->17059839] (req 17055744->17059839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.415435:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507159936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 101353us (101851us total) trans 4294971484 rc 0/0 00000100:00100000:2.0:1789760708.415446:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4194 00000100:00100000:2.0:1789760708.415455:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689503348783 time 689629360734 state 0x2 ref 0 00000100:00100000:1.0:1789760708.423913:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.424010:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160064 00000100:00100000:2.0:1789760708.424019:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160064 00000100:00100000:2.0:1789760708.424086:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689503348783 time 689629360734 state 0x0 ref 1 00000100:00100000:2.0:1789760708.424101:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689629621150 time 689629360734 state 0x1 ref 1 00000100:00100000:2.0:1789760708.424126:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689629621150 time 689739429401 state 0x0 ref 1 00000100:00100000:2.0:1789760708.424136:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507160064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.424190:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17059840, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:2.0:1789760708.424198:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.424214:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760708.424219:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.424225:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.424239:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508428:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760708.519403:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17059840 -> 17063936 00010000:00010000:2.0:1789760708.519422:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519436:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519673:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519683:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.519688:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519725:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519745:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.519753:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17059840->17063935] (req 17059840->17063935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.519951:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971485 is committed 00000100:00100000:2.0:1789760708.520214:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507160064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96073us (96304us total) trans 4294971485 rc 0/0 00000100:00100000:2.0:1789760708.520225:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4195 00000100:00100000:2.0:1789760708.520234:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689629621150 time 689739429401 state 0x2 ref 0 00000100:00100000:0.0:1789760708.527302:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.527395:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160192 00000100:00100000:2.0:1789760708.527404:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160192 00000100:00100000:2.0:1789760708.527460:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689629621150 time 689739429401 state 0x0 ref 1 00000100:00100000:2.0:1789760708.527475:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689739689817 time 689739429401 state 0x1 ref 1 00000100:00100000:2.0:1789760708.527603:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689739689817 time 689842905918 state 0x0 ref 1 00000100:00100000:2.0:1789760708.527616:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507160192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.527662:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17063936, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:2.0:1789760708.527671:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.527685:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760708.527689:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.527725:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.527740:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.575596:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971486 is committed 00002000:00010000:3.0:1789760708.575754:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17063936 -> 17068032 00010000:00010000:3.0:1789760708.575769:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575781:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575791:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575799:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760708.575804:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575814:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575839:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.575847:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17063936->17068031] (req 17063936->17068031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.575982:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507160192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48363us (48685us total) trans 4294971486 rc 0/0 00000100:00100000:3.0:1789760708.575992:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4196 00000100:00100000:3.0:1789760708.576002:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689739689817 time 689842905918 state 0x2 ref 0 00000100:00100000:0.0:1789760708.579685:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.580266:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160320 00000100:00100000:2.0:1789760708.580277:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160320 00000100:00100000:2.0:1789760708.580333:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689739689817 time 689842905918 state 0x0 ref 1 00000100:00100000:2.0:1789760708.580348:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689843166334 time 689842905918 state 0x1 ref 1 00000100:00100000:2.0:1789760708.580377:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689843166334 time 689895680998 state 0x0 ref 1 00000100:00100000:2.0:1789760708.580389:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.580435:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17068032, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:2.0:1789760708.580443:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.580457:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760708.580461:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.580467:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.580539:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.621314:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971487 is committed 00002000:00010000:2.0:1789760708.621465:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17068032 -> 17072128 00010000:00010000:2.0:1789760708.621490:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621504:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621514:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621522:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.621527:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621536:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621554:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.621562:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6e2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17068032->17072127] (req 17068032->17072127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.621656:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41264us (41974us total) trans 4294971487 rc 0/0 00000100:00100000:2.0:1789760708.621667:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4197 00000100:00100000:2.0:1789760708.621677:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689843166334 time 689895680998 state 0x2 ref 0 00000100:00100000:3.0:1789760708.625089:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.625538:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160448 00000100:00100000:2.0:1789760708.625548:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160448 00000100:00100000:2.0:1789760708.625605:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689843166334 time 689895680998 state 0x0 ref 1 00000100:00100000:2.0:1789760708.625619:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689895941414 time 689895680998 state 0x1 ref 1 00000100:00100000:2.0:1789760708.625649:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689895941414 time 689940951962 state 0x0 ref 1 00000100:00100000:2.0:1789760708.625664:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.625733:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17072128, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:2.0:1789760708.625742:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.625758:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760708.625763:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.625769:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.625782:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.664891:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17072128 -> 17076224 00010000:00010000:2.0:1789760708.664912:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664925:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664933:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664940:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.664945:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664954:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664985:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.664993:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17072128->17076223] (req 17072128->17076223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.665113:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507160448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39446us (40039us total) trans 4294971488 rc 0/0 00000100:00100000:2.0:1789760708.665123:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4198 00000100:00100000:2.0:1789760708.665132:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689895941414 time 689940951962 state 0x2 ref 0 00000001:00080000:2.0:1789760708.668982:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971488 is committed 00000100:00100000:2.0:1789760708.670218:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760708.670625:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160576 00000100:00100000:3.0:1789760708.670636:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160576 00000100:00100000:3.0:1789760708.670726:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689895941414 time 689940951962 state 0x0 ref 1 00000100:00100000:3.0:1789760708.670743:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689941212378 time 689940951962 state 0x1 ref 1 00000100:00100000:3.0:1789760708.670770:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689941212378 time 689986073549 state 0x0 ref 1 00000100:00100000:3.0:1789760708.670782:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760708.670830:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17076224, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:3.0:1789760708.670840:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.670862:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760708.670866:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.670873:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.670888:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.723858:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971489 is committed 00002000:00010000:1.0:1789760708.724898:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17076224 -> 17080320 00010000:00010000:1.0:1789760708.724915:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.724935:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.724945:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.724953:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760708.725347:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.725361:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.725384:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.725391:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6e3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17076224->17080319] (req 17076224->17080319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.726026:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55240us (55809us total) trans 4294971489 rc 0/0 00000100:00100000:1.0:1789760708.726046:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4199 00000100:00100000:1.0:1789760708.726056:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689941212378 time 689986073549 state 0x2 ref 0 00000100:00100000:0.0:1789760708.732650:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.733054:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160704 00000100:00100000:2.0:1789760708.733063:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160704 00000100:00100000:2.0:1789760708.733271:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689941212378 time 689986073549 state 0x0 ref 1 00000100:00100000:2.0:1789760708.733284:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689986333965 time 689986073549 state 0x1 ref 1 00000100:00100000:2.0:1789760708.733348:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689986333965 time 690048650695 state 0x0 ref 1 00000100:00100000:2.0:1789760708.733360:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507160704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760708.733408:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17080320, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:2.0:1789760708.733416:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.733431:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760708.733435:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.733441:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.733454:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.781938:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17080320 -> 17084416 00010000:00010000:2.0:1789760708.781956:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782041:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782052:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782063:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.782272:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782285:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782306:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.782314:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17080320->17084415] (req 17080320->17084415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.782400:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507160704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49037us (49755us total) trans 4294971490 rc 0/0 00000100:00100000:2.0:1789760708.782411:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4200 00000100:00100000:2.0:1789760708.782420:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689986333965 time 690048650695 state 0x2 ref 0 00000001:00080000:3.0:1789760708.782606:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971490 is committed 00000100:00100000:0.0:1789760708.790779:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760708.790869:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160832 00000100:00100000:3.0:1789760708.790880:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160832 00000100:00100000:3.0:1789760708.790936:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 689986333965 time 690048650695 state 0x0 ref 1 00000100:00100000:3.0:1789760708.790954:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690048911111 time 690048650695 state 0x1 ref 1 00000100:00100000:3.0:1789760708.790990:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690048911111 time 690106293521 state 0x0 ref 1 00000100:00100000:3.0:1789760708.791004:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760708.791063:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17084416, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:3.0:1789760708.791088:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.791104:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760708.791108:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.791114:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.791128:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.829539:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105822d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538259200:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760708.829604:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760708.833132:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259200 00000100:00100000:2.0:1789760708.833153:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259200 00000100:00100000:2.0:1789760708.833226:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1789760708.833235:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538259200:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760708.833264:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3840933888 avail 3837788160 00002000:00010000:3.0:1789760708.834330:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17084416 -> 17088512 00010000:00010000:3.0:1789760708.834346:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834358:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834368:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834375:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760708.834380:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834389:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834408:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.834417:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17084416->17088511] (req 17084416->17088511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.834507:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507160832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43502us (43734us total) trans 4294971491 rc 0/0 00000100:00100000:3.0:1789760708.834517:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4201 00000100:00100000:3.0:1789760708.834527:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690048911111 time 690106293521 state 0x2 ref 0 00000001:00080000:3.0:1789760708.834561:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971491 is committed 00000100:00100000:3.0:1789760708.838855:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.839473:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538259200:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6236us (9865us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760708.839553:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1789760708.839674:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538259328:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760708.839746:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760708.839807:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507160960 00000100:00100000:1.0:1789760708.839815:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507160960 00000100:00100000:1.0:1789760708.839878:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690048911111 time 690106293521 state 0x0 ref 1 00000100:00100000:1.0:1789760708.839895:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690106553937 time 690106293521 state 0x1 ref 1 00000100:00100000:1.0:1789760708.839917:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690106553937 time 690155220698 state 0x0 ref 1 00000100:00100000:1.0:1789760708.839929:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507160960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760708.840470:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17088512, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:1.0:1789760708.840490:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.840506:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760708.840511:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.840518:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.840532:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.840926:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259328 00000100:00100000:1.0:1789760708.840932:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259328 00000100:00100000:1.0:1789760708.841351:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:1.0:1789760708.841362:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538259328:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760708.841381:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3840933888 avail 3836739584 00000100:00100000:0.0:1789760708.841540:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105822d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538259200:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760708.841818:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538259328:12345-0@lo:13:osp-pre-0-0.0 Request processed in 458us (2070us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760708.841839:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1789760708.842293:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538259328:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760708.874548:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17088512 -> 17092608 00010000:00010000:2.0:1789760708.874570:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874584:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874594:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874601:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760708.874607:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874615:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874635:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.874642:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6e4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17088512->17092607] (req 17088512->17092607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.874758:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507160960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34826us (35906us total) trans 4294971492 rc 0/0 00000100:00100000:2.0:1789760708.874768:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4202 00000100:00100000:2.0:1789760708.874777:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690106553937 time 690155220698 state 0x2 ref 0 00000001:00080000:0.0:1789760708.874931:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971492 is committed 00000100:00100000:3.0:1789760708.883581:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760708.886658:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161088 00000100:00100000:2.0:1789760708.886672:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161088 00000100:00100000:2.0:1789760708.886747:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690106553937 time 690155220698 state 0x0 ref 1 00000100:00100000:2.0:1789760708.886763:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690155481114 time 690155220698 state 0x1 ref 1 00000100:00100000:3.0:1789760708.887305:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690155481114 time 690202605140 state 0x0 ref 1 00000100:00100000:3.0:1789760708.887323:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760708.887377:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17092608, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:3.0:1789760708.887386:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.887403:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760708.887407:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.887414:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.887429:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.006295:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971493 is committed 00002000:00010000:2.0:1789760709.011291:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17092608 -> 17096704 00010000:00010000:2.0:1789760709.011312:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011326:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011335:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011342:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.011346:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011355:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011375:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.011383:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6e56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17092608->17096703] (req 17092608->17096703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.011538:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 124215us (129579us total) trans 4294971493 rc 0/0 00000100:00100000:2.0:1789760709.011548:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4203 00000100:00100000:2.0:1789760709.011558:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690155481114 time 690202605140 state 0x2 ref 0 00000100:00100000:3.0:1789760709.027383:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760709.027485:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161216 00000100:00100000:1.0:1789760709.027494:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161216 00000100:00100000:1.0:1789760709.027565:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690155481114 time 690202605140 state 0x0 ref 1 00000100:00100000:1.0:1789760709.027579:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690202865556 time 690202605140 state 0x1 ref 1 00000100:00100000:1.0:1789760709.027607:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690202865556 time 690342910830 state 0x0 ref 1 00000100:00100000:1.0:1789760709.027620:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507161216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760709.027666:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17096704, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x8807 00010000:00010000:1.0:1789760709.027674:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.027689:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760709.027718:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.027725:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.027739:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760709.031594:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760709.075551:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971494 is committed 00002000:00010000:3.0:1789760709.075636:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17096704 -> 17100800 00002000:00010000:3.0:1789760709.075640:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:3.0:1789760709.075643:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:3.0:1789760709.075656:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.075669:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.075678:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.075685:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760709.075690:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.076018:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.076047:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.076055:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b6e5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17096704->17100799] (req 17096704->17100799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.076371:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507161216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48746us (48990us total) trans 4294971494 rc 0/0 00000100:00100000:2.0:1789760709.076382:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4204 00000100:00100000:2.0:1789760709.076391:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690202865556 time 690342910830 state 0x2 ref 0 00000100:00100000:3.0:1789760709.079605:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760709.079841:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161344 00000100:00100000:0.0:1789760709.079851:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161344 00000100:00100000:0.0:1789760709.079899:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690202865556 time 690342910830 state 0x0 ref 1 00000100:00100000:0.0:1789760709.079913:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690343171246 time 690342910830 state 0x1 ref 1 00000100:00100000:0.0:1789760709.079939:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690343171246 time 690395242999 state 0x0 ref 1 00000100:00100000:0.0:1789760709.079949:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507161344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760709.079996:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17100800, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:0.0:1789760709.080005:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.080020:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760709.080024:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.080038:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.080053:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.131666:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971495 is committed 00002000:00010000:0.0:1789760709.131848:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17100800 -> 17104896 00010000:00010000:0.0:1789760709.131865:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.131878:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.131893:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.131899:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760709.131903:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.131911:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.132187:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.132197:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b6e64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17100800->17104895] (req 17100800->17104895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.134259:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507161344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54298us (54651us total) trans 4294971495 rc 0/0 00000100:00100000:0.0:1789760709.134274:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4205 00000100:00100000:0.0:1789760709.134283:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690343171246 time 690395242999 state 0x2 ref 0 00000100:00100000:3.0:1789760709.139334:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760709.139945:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161472 00000100:00100000:2.0:1789760709.139955:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161472 00000100:00100000:2.0:1789760709.140011:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690343171246 time 690395242999 state 0x0 ref 1 00000100:00100000:2.0:1789760709.140028:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690395503415 time 690395242999 state 0x1 ref 1 00000100:00100000:2.0:1789760709.140079:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690395503415 time 690455381999 state 0x0 ref 1 00000100:00100000:2.0:1789760709.140100:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760709.140149:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17104896, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:2.0:1789760709.140158:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.140173:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760709.140177:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.140184:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.140199:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760709.211455:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17104896 -> 17108992 00010000:00010000:0.0:1789760709.211483:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211501:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211509:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211516:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760709.211520:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211529:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211549:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211556:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6e6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17104896->17108991] (req 17104896->17108991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.211675:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507161472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71572us (72344us total) trans 4294971496 rc 0/0 00000100:00100000:0.0:1789760709.211686:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4206 00000100:00100000:0.0:1789760709.211734:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690395503415 time 690455381999 state 0x2 ref 0 00000001:00080000:1.0:1789760709.215492:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971496 is committed 00000100:00100000:2.0:1789760709.218495:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760709.218631:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161600 00000100:00100000:1.0:1789760709.218641:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161600 00000100:00100000:1.0:1789760709.218690:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690395503415 time 690455381999 state 0x0 ref 1 00000100:00100000:1.0:1789760709.218731:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690455642415 time 690455381999 state 0x1 ref 1 00000100:00100000:1.0:1789760709.218758:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690455642415 time 690534061077 state 0x0 ref 1 00000100:00100000:1.0:1789760709.218770:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507161600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760709.218815:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17108992, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:1.0:1789760709.218826:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.218840:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760709.218844:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.218850:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.218863:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760709.276893:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17108992 -> 17113088 00010000:00010000:1.0:1789760709.276913:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276930:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276938:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276946:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760709.276951:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276968:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276987:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.276994:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17108992->17113087] (req 17108992->17113087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.277193:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507161600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58420us (58712us total) trans 4294971497 rc 0/0 00000100:00100000:1.0:1789760709.277205:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4207 00000100:00100000:1.0:1789760709.277214:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690455642415 time 690534061077 state 0x2 ref 0 00000001:00080000:3.0:1789760709.280900:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971497 is committed 00000100:00100000:2.0:1789760709.281230:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760709.281303:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161728 00000100:00100000:0.0:1789760709.281313:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161728 00000100:00100000:0.0:1789760709.281365:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690455642415 time 690534061077 state 0x0 ref 1 00000100:00100000:0.0:1789760709.281381:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690534321493 time 690534061077 state 0x1 ref 1 00000100:00100000:0.0:1789760709.281444:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690534321493 time 690596746588 state 0x0 ref 1 00000100:00100000:0.0:1789760709.281455:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760709.281595:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17113088, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:0.0:1789760709.281612:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.281626:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760709.281630:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.281637:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.281650:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.326939:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971498 is committed 00002000:00010000:2.0:1789760709.327216:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17113088 -> 17117184 00010000:00010000:2.0:1789760709.327232:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327245:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327255:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327262:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.327266:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327275:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327293:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327300:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b6e79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17113088->17117183] (req 17113088->17117183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.327494:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46034us (46265us total) trans 4294971498 rc 0/0 00000100:00100000:2.0:1789760709.327504:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4208 00000100:00100000:2.0:1789760709.327514:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690534321493 time 690596746588 state 0x2 ref 0 00000100:00100000:0.0:1789760709.332670:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760709.332795:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161856 00000100:00100000:1.0:1789760709.332806:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161856 00000100:00100000:1.0:1789760709.332863:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690534321493 time 690596746588 state 0x0 ref 1 00000100:00100000:1.0:1789760709.332878:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690597007004 time 690596746588 state 0x1 ref 1 00000100:00100000:1.0:1789760709.332964:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690597007004 time 690648256889 state 0x0 ref 1 00000100:00100000:1.0:1789760709.332977:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507161856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760709.333037:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17117184, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:1.0:1789760709.333047:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.333069:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760709.333074:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.333236:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.333253:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.340212:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760709.340235:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538259456:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760709.340303:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760709.340412:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259456 00000100:00100000:1.0:1789760709.340420:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259456 00000100:00100000:1.0:1789760709.340469:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789760709.340484:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538259456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760709.340517:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760709.340558:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538259456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760709.340567:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789760709.340766:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538259456:0@lo:41:osp-pre-1-0.0 00002000:00010000:2.0:1789760709.392832:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17117184 -> 17121280 00010000:00010000:2.0:1789760709.392852:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392866:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392875:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392882:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.392886:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392895:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392913:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.392924:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b6e80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17117184->17121279] (req 17117184->17121279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.393022:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507161856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60042us (60420us total) trans 4294971499 rc 0/0 00000100:00100000:2.0:1789760709.393039:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4209 00000100:00100000:2.0:1789760709.393048:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690597007004 time 690648256889 state 0x2 ref 0 00000001:00080000:3.0:1789760709.395271:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971499 is committed 00000100:00100000:3.0:1789760709.400790:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760709.400909:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507161984 00000100:00100000:1.0:1789760709.400919:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507161984 00000100:00100000:1.0:1789760709.400969:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690597007004 time 690648256889 state 0x0 ref 1 00000100:00100000:1.0:1789760709.400984:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690648517305 time 690648256889 state 0x1 ref 1 00000100:00100000:1.0:1789760709.401016:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690648517305 time 690716319032 state 0x0 ref 1 00000100:00100000:1.0:1789760709.401037:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507161984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760709.401238:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17121280, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:1.0:1789760709.401247:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.401263:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760709.401267:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.401274:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.401288:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.478737:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971500 is committed 00002000:00010000:2.0:1789760709.478865:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17121280 -> 17125376 00010000:00010000:2.0:1789760709.478880:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478894:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478903:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478909:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.478914:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478922:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478947:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.478954:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b6e87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17121280->17125375] (req 17121280->17125375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.479066:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507161984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78024us (78270us total) trans 4294971500 rc 0/0 00000100:00100000:2.0:1789760709.479364:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4210 00000100:00100000:2.0:1789760709.479378:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690648517305 time 690716319032 state 0x2 ref 0 00000100:00100000:1.0:1789760709.483458:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760709.483592:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162112 00000100:00100000:2.0:1789760709.483664:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162112 00000100:00100000:2.0:1789760709.483756:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690648517305 time 690716319032 state 0x0 ref 1 00000100:00100000:2.0:1789760709.483773:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690716579448 time 690716319032 state 0x1 ref 1 00000100:00100000:2.0:1789760709.483803:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690716579448 time 690799106736 state 0x0 ref 1 00000100:00100000:2.0:1789760709.483816:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507162112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760709.483865:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17125376, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:2.0:1789760709.483873:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.483889:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760709.483894:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.483901:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.483913:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.532483:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760709.536929:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971501 is committed 00002000:00010000:2.0:1789760709.537045:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17125376 -> 17129472 00010000:00010000:2.0:1789760709.537243:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537255:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537263:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537270:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.537275:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537285:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537304:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.537312:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17125376->17129471] (req 17125376->17129471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.537580:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507162112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53760us (54125us total) trans 4294971501 rc 0/0 00000100:00100000:2.0:1789760709.537592:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4211 00000100:00100000:2.0:1789760709.537668:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690716579448 time 690799106736 state 0x2 ref 0 00000100:00100000:0.0:1789760709.544410:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760709.544510:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162240 00000100:00100000:2.0:1789760709.544521:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162240 00000100:00100000:2.0:1789760709.544577:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690716579448 time 690799106736 state 0x0 ref 1 00000100:00100000:2.0:1789760709.544593:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690799367152 time 690799106736 state 0x1 ref 1 00000100:00100000:2.0:1789760709.544633:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690799367152 time 690859936558 state 0x0 ref 1 00000100:00100000:2.0:1789760709.544646:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507162240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760709.544690:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17129472, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:2.0:1789760709.544725:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.544739:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760709.544743:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.544749:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.544761:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.594000:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971502 is committed 00002000:00010000:2.0:1789760709.594121:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17129472 -> 17133568 00010000:00010000:2.0:1789760709.594137:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594150:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594158:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594165:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.594170:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594178:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594199:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.594207:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b6e95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17129472->17133567] (req 17129472->17133567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.594298:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507162240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49649us (49892us total) trans 4294971502 rc 0/0 00000100:00100000:2.0:1789760709.594312:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4212 00000100:00100000:2.0:1789760709.594322:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690799367152 time 690859936558 state 0x2 ref 0 00000100:00100000:0.0:1789760709.599171:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760709.599253:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162368 00000100:00100000:2.0:1789760709.599263:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162368 00000100:00100000:2.0:1789760709.599317:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690799367152 time 690859936558 state 0x0 ref 1 00000100:00100000:2.0:1789760709.599331:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690860196974 time 690859936558 state 0x1 ref 1 00000100:00100000:2.0:1789760709.599743:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690860196974 time 690915044650 state 0x0 ref 1 00000100:00100000:2.0:1789760709.599756:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507162368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760709.599804:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17133568, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:2.0:1789760709.599813:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.599827:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760709.599831:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.599837:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.599851:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.649854:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17133568 -> 17137664 00010000:00010000:2.0:1789760709.649873:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649886:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649895:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649902:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.649907:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649916:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649935:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.649942:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b6e9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17133568->17137663] (req 17133568->17137663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.650045:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507162368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50287us (50880us total) trans 4294971503 rc 0/0 00000100:00100000:2.0:1789760709.650954:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4213 00000100:00100000:2.0:1789760709.650968:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690860196974 time 690915044650 state 0x2 ref 0 00000001:00080000:0.0:1789760709.654134:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971503 is committed 00000100:00100000:2.0:1789760709.656958:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760709.657685:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162496 00000100:00100000:0.0:1789760709.657731:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162496 00000100:00100000:0.0:1789760709.657787:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690860196974 time 690915044650 state 0x0 ref 1 00000100:00100000:0.0:1789760709.657803:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690915305066 time 690915044650 state 0x1 ref 1 00000100:00100000:0.0:1789760709.657832:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690915305066 time 690973135124 state 0x0 ref 1 00000100:00100000:0.0:1789760709.657845:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507162496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760709.658332:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17137664, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:0.0:1789760709.658343:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658359:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760709.658364:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.658371:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658457:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.736608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971504 is committed 00002000:00010000:3.0:1789760709.736825:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17137664 -> 17141760 00010000:00010000:3.0:1789760709.736841:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736853:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736863:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736869:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760709.736874:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736883:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736905:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.736914:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b6ea3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17137664->17141759] (req 17137664->17141759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.737008:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507162496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79160us (80056us total) trans 4294971504 rc 0/0 00000100:00100000:3.0:1789760709.737019:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4214 00000100:00100000:3.0:1789760709.737042:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690915305066 time 690973135124 state 0x2 ref 0 00000100:00100000:3.0:1789760709.741849:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760709.741893:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162624 00000100:00100000:3.0:1789760709.741902:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162624 00000100:00100000:3.0:1789760709.741952:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690915305066 time 690973135124 state 0x0 ref 1 00000100:00100000:3.0:1789760709.741967:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690973395540 time 690973135124 state 0x1 ref 1 00000100:00100000:3.0:1789760709.741989:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690973395540 time 691057293025 state 0x0 ref 1 00000100:00100000:3.0:1789760709.742001:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507162624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760709.742070:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17141760, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:3.0:1789760709.742085:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.742101:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760709.742106:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.742112:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.742126:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.822053:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971505 is committed 00002000:00010000:0.0:1789760709.822389:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17141760 -> 17145856 00010000:00010000:0.0:1789760709.822405:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822416:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822431:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822439:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760709.822444:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822453:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822474:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.822482:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6eaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17141760->17145855] (req 17141760->17145855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.822569:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507162624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 80566us (80725us total) trans 4294971505 rc 0/0 00000100:00100000:0.0:1789760709.822580:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4215 00000100:00100000:0.0:1789760709.822588:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690973395540 time 691057293025 state 0x2 ref 0 00000100:00100000:2.0:1789760709.825960:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760709.826062:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162752 00000100:00100000:1.0:1789760709.826073:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162752 00000100:00100000:1.0:1789760709.826127:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 690973395540 time 691057293025 state 0x0 ref 1 00000100:00100000:1.0:1789760709.826143:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691057553441 time 691057293025 state 0x1 ref 1 00000100:00100000:1.0:1789760709.826174:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691057553441 time 691141476801 state 0x0 ref 1 00000100:00100000:1.0:1789760709.826186:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507162752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760709.826236:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17145856, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:1.0:1789760709.826245:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.826261:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760709.826265:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.826271:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.826284:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.858176:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538259584:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760709.858241:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760709.859051:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259584 00000100:00100000:1.0:1789760709.859073:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259584 00000100:00100000:1.0:1789760709.859229:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789760709.859238:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538259584:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760709.859277:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760709.859681:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538259584:12345-0@lo:41:osp-pre-0-1.0 Request processed in 443us (1438us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760709.859689:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789760709.859931:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538259584:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760709.905899:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971506 is committed 00002000:00010000:2.0:1789760709.907210:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17145856 -> 17149952 00010000:00010000:2.0:1789760709.907228:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907242:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907252:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907259:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.907264:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907272:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907293:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.907301:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b6eb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17145856->17149951] (req 17145856->17149951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.907402:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507162752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 81212us (81446us total) trans 4294971506 rc 0/0 00000100:00100000:2.0:1789760709.907413:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4216 00000100:00100000:2.0:1789760709.907422:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691057553441 time 691141476801 state 0x2 ref 0 00000100:00100000:2.0:1789760709.916675:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760709.916873:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507162880 00000100:00100000:3.0:1789760709.916883:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507162880 00000100:00100000:3.0:1789760709.916932:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691057553441 time 691141476801 state 0x0 ref 1 00000100:00100000:3.0:1789760709.916947:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691141737217 time 691141476801 state 0x1 ref 1 00000100:00100000:3.0:1789760709.916981:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691141737217 time 691232284210 state 0x0 ref 1 00000100:00100000:3.0:1789760709.916992:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507162880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760709.917047:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17149952, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:3.0:1789760709.917056:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.917080:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760709.917084:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.917090:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.917103:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.981844:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971507 is committed 00002000:00010000:2.0:1789760709.981972:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17149952 -> 17154048 00010000:00010000:2.0:1789760709.981987:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982000:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982009:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982017:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760709.982022:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982041:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982069:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.982082:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6eb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17149952->17154047] (req 17149952->17154047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.982177:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507162880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65181us (65506us total) trans 4294971507 rc 0/0 00000100:00100000:2.0:1789760709.982187:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4217 00000100:00100000:2.0:1789760709.982197:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691141737217 time 691232284210 state 0x2 ref 0 00000100:00100000:0.0:1789760709.994828:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760709.994917:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163008 00000100:00100000:3.0:1789760709.994928:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163008 00000100:00100000:3.0:1789760709.994980:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691141737217 time 691232284210 state 0x0 ref 1 00000100:00100000:3.0:1789760709.994996:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691232544626 time 691232284210 state 0x1 ref 1 00000100:00100000:3.0:1789760709.995143:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691232544626 time 691310444575 state 0x0 ref 1 00000100:00100000:3.0:1789760709.995157:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507163008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760709.995209:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17154048, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:3.0:1789760709.995218:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.995234:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760709.995239:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.995245:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.995259:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.092107:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971508 is committed 00002000:00010000:0.0:1789760710.092448:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17154048 -> 17158144 00010000:00010000:0.0:1789760710.092464:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092477:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092486:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092492:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760710.092497:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092506:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092525:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.092533:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b6ebf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17154048->17158143] (req 17154048->17158143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.092674:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507163008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 97514us (97851us total) trans 4294971508 rc 0/0 00000100:00100000:0.0:1789760710.092684:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4218 00000100:00100000:0.0:1789760710.092742:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691232544626 time 691310444575 state 0x2 ref 0 00000100:00100000:0.0:1789760710.120477:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760710.120573:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163136 00000100:00100000:2.0:1789760710.120583:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163136 00000100:00100000:2.0:1789760710.120638:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691232544626 time 691310444575 state 0x0 ref 1 00000100:00100000:2.0:1789760710.120653:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691310704991 time 691310444575 state 0x1 ref 1 00000100:00100000:2.0:1789760710.120683:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691310704991 time 691435986469 state 0x0 ref 1 00000100:00100000:2.0:1789760710.120730:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507163136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760710.120778:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17158144, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x8807 00010000:00010000:2.0:1789760710.120787:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.120803:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760710.120807:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.120814:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.120834:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.192403:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971509 is committed 00002000:00010000:2.0:1789760710.192587:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17158144 -> 17162240 00002000:00010000:2.0:1789760710.192592:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760708 -> 1789760709 00002000:00010000:2.0:1789760710.192595:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760708 -> 1789760709 00010000:00010000:2.0:1789760710.192608:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192621:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192630:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192638:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760710.192644:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192653:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192673:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.192686:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6ec6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17158144->17162239] (req 17158144->17162239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.192795:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507163136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72097us (72323us total) trans 4294971509 rc 0/0 00000100:00100000:2.0:1789760710.192805:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4219 00000100:00100000:2.0:1789760710.192815:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691310704991 time 691435986469 state 0x2 ref 0 00000100:00100000:2.0:1789760710.199292:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760710.205258:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163264 00000100:00100000:3.0:1789760710.205275:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163264 00000100:00100000:3.0:1789760710.205341:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691310704991 time 691435986469 state 0x0 ref 1 00000100:00100000:3.0:1789760710.205361:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691436246885 time 691435986469 state 0x1 ref 1 00000100:00100000:3.0:1789760710.205399:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691436246885 time 691520701327 state 0x0 ref 1 00000100:00100000:3.0:1789760710.205414:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507163264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760710.205462:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17162240, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760710.205470:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.205485:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760710.205489:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.205495:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.205509:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.358745:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17162240 -> 17166336 00010000:00010000:3.0:1789760710.358764:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358778:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358786:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358794:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760710.358798:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358807:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358827:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.358836:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ecd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17162240->17166335] (req 17162240->17166335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.358920:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507163264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 153505us (159634us total) trans 4294971510 rc 0/0 00000100:00100000:3.0:1789760710.358930:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4220 00000100:00100000:3.0:1789760710.358939:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691436246885 time 691520701327 state 0x2 ref 0 00000001:00080000:3.0:1789760710.358988:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971510 is committed 00000100:00100000:0.0:1789760710.369881:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760710.370256:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163392 00000100:00100000:3.0:1789760710.370266:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163392 00000100:00100000:3.0:1789760710.370529:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691436246885 time 691520701327 state 0x0 ref 1 00000100:00100000:3.0:1789760710.370547:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691520961743 time 691520701327 state 0x1 ref 1 00000100:00100000:3.0:1789760710.370615:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691520961743 time 691685918052 state 0x0 ref 1 00000100:00100000:3.0:1789760710.370628:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760710.370675:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17166336, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760710.370684:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.370717:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760710.370722:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.370729:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.370744:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.456238:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971511 is committed 00002000:00010000:2.0:1789760710.456344:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17166336 -> 17170432 00010000:00010000:2.0:1789760710.456358:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456370:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456379:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456396:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760710.456402:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456412:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456436:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.456443:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b6ed4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17166336->17170431] (req 17166336->17170431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.456537:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 85906us (86660us total) trans 4294971511 rc 0/0 00000100:00100000:2.0:1789760710.456547:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4221 00000100:00100000:2.0:1789760710.456555:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691520961743 time 691685918052 state 0x2 ref 0 00000100:00100000:0.0:1789760710.460122:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760710.460231:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163520 00000100:00100000:2.0:1789760710.460240:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163520 00000100:00100000:2.0:1789760710.460286:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691520961743 time 691685918052 state 0x0 ref 1 00000100:00100000:2.0:1789760710.460301:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691686178468 time 691685918052 state 0x1 ref 1 00000100:00100000:2.0:1789760710.460328:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691686178468 time 691775631926 state 0x0 ref 1 00000100:00100000:2.0:1789760710.460340:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760710.460398:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17170432, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:2.0:1789760710.460406:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.460422:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760710.460428:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.460434:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.460448:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.542344:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971512 is committed 00002000:00010000:2.0:1789760710.542524:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17170432 -> 17174528 00010000:00010000:2.0:1789760710.542541:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542553:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542561:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542568:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760710.542573:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542588:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542613:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.542624:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b6edb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17170432->17174527] (req 17170432->17174527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.542750:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82407us (82632us total) trans 4294971512 rc 0/0 00000100:00100000:2.0:1789760710.542761:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4222 00000100:00100000:2.0:1789760710.542770:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691686178468 time 691775631926 state 0x2 ref 0 00000100:00100000:3.0:1789760710.547442:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760710.547606:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163648 00000100:00100000:0.0:1789760710.547616:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163648 00000100:00100000:0.0:1789760710.547673:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691686178468 time 691775631926 state 0x0 ref 1 00000100:00100000:0.0:1789760710.547688:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691775892342 time 691775631926 state 0x1 ref 1 00000100:00100000:0.0:1789760710.547756:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691775892342 time 691863058732 state 0x0 ref 1 00000100:00100000:0.0:1789760710.547768:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760710.547816:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17174528, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:0.0:1789760710.547825:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.547841:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760710.547846:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.547852:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.547866:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.556144:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760710.673944:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17174528 -> 17178624 00010000:00010000:0.0:1789760710.673968:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.673983:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.673992:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.674000:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760710.674005:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.674014:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.674045:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.674061:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ee2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17174528->17178623] (req 17174528->17178623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.674157:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507163648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 126385us (126719us total) trans 4294971513 rc 0/0 00000100:00100000:0.0:1789760710.674168:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4223 00000100:00100000:0.0:1789760710.674178:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691775892342 time 691863058732 state 0x2 ref 0 00000001:00080000:3.0:1789760710.675238:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971513 is committed 00000100:00100000:2.0:1789760710.686868:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760710.687014:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163776 00000100:00100000:3.0:1789760710.687034:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163776 00000100:00100000:3.0:1789760710.687100:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691775892342 time 691863058732 state 0x0 ref 1 00000100:00100000:3.0:1789760710.687126:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691863319148 time 691863058732 state 0x1 ref 1 00000100:00100000:3.0:1789760710.687283:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691863319148 time 692002585528 state 0x0 ref 1 00000100:00100000:3.0:1789760710.687298:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760710.687343:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17178624, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760710.687351:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.687372:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760710.687377:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.687788:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.687807:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.769443:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971514 is committed 00002000:00010000:3.0:1789760710.769660:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17178624 -> 17182720 00010000:00010000:3.0:1789760710.769676:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769689:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769720:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769728:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760710.769733:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769742:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769761:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.769769:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6ee9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17178624->17182719] (req 17178624->17182719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.769853:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82554us (82991us total) trans 4294971514 rc 0/0 00000100:00100000:3.0:1789760710.769863:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4224 00000100:00100000:3.0:1789760710.769872:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691863319148 time 692002585528 state 0x2 ref 0 00000100:00100000:1.0:1789760710.796610:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760710.796715:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507163904 00000100:00100000:2.0:1789760710.796724:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507163904 00000100:00100000:2.0:1789760710.796778:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 691863319148 time 692002585528 state 0x0 ref 1 00000100:00100000:2.0:1789760710.796795:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692002845944 time 692002585528 state 0x1 ref 1 00000100:00100000:2.0:1789760710.796824:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692002845944 time 692112127347 state 0x0 ref 1 00000100:00100000:2.0:1789760710.796836:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760710.796882:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17182720, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:2.0:1789760710.796890:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.796906:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760710.796910:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.796917:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.796931:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.860252:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971515 is committed 00002000:00010000:3.0:1789760710.862208:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17182720 -> 17186816 00010000:00010000:3.0:1789760710.862227:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862240:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862249:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862256:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760710.862261:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862270:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862294:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.862302:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b6ef0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17182720->17186815] (req 17182720->17186815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.862398:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507163904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65559us (65793us total) trans 4294971515 rc 0/0 00000100:00100000:3.0:1789760710.862409:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4225 00000100:00100000:3.0:1789760710.862418:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692002845944 time 692112127347 state 0x2 ref 0 00000100:00100000:2.0:1789760710.865486:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760710.865792:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164032 00000100:00100000:0.0:1789760710.865802:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164032 00000100:00100000:0.0:1789760710.865853:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692002845944 time 692112127347 state 0x0 ref 1 00000100:00100000:0.0:1789760710.865867:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692112387763 time 692112127347 state 0x1 ref 1 00000100:00100000:0.0:1789760710.865897:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692112387763 time 692181201158 state 0x0 ref 1 00000100:00100000:0.0:1789760710.865908:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760710.865951:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17186816, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:0.0:1789760710.865960:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.865974:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760710.865979:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.865985:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.866000:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.904246:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971516 is committed 00002000:00010000:3.0:1789760710.904526:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17186816 -> 17190912 00010000:00010000:3.0:1789760710.904541:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904553:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904561:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904567:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760710.904571:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904583:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904601:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.904607:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b6ef7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17186816->17190911] (req 17186816->17190911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.904726:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38783us (39212us total) trans 4294971516 rc 0/0 00000100:00100000:3.0:1789760710.904736:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4226 00000100:00100000:3.0:1789760710.904745:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692112387763 time 692181201158 state 0x2 ref 0 00000100:00100000:2.0:1789760710.917866:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760710.918095:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164160 00000100:00100000:3.0:1789760710.918107:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164160 00000100:00100000:3.0:1789760710.918542:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692112387763 time 692181201158 state 0x0 ref 1 00000100:00100000:3.0:1789760710.918558:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692181461574 time 692181201158 state 0x1 ref 1 00000100:00100000:3.0:1789760710.918650:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692181461574 time 692233951454 state 0x0 ref 1 00000100:00100000:3.0:1789760710.918665:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760710.918757:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17190912, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760710.918766:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.918781:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760710.918785:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.918791:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.918804:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.946943:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971517 is committed 00002000:00010000:3.0:1789760710.947104:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17190912 -> 17195008 00010000:00010000:3.0:1789760710.947120:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947132:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947141:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947148:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760710.947153:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947162:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947182:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.947191:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6efe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17190912->17195007] (req 17190912->17195007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.947285:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28619us (29423us total) trans 4294971517 rc 0/0 00000100:00100000:3.0:1789760710.947297:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4227 00000100:00100000:3.0:1789760710.947317:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692181461574 time 692233951454 state 0x2 ref 0 00000100:00100000:0.0:1789760710.963049:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760710.964026:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164288 00000100:00100000:3.0:1789760710.964038:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164288 00000100:00100000:3.0:1789760710.964096:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692181461574 time 692233951454 state 0x0 ref 1 00000100:00100000:3.0:1789760710.964111:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692234211870 time 692233951454 state 0x1 ref 1 00000100:00100000:3.0:1789760710.964146:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692234211870 time 692279448369 state 0x0 ref 1 00000100:00100000:3.0:1789760710.964160:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760710.964210:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17195008, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760710.964220:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.964235:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760710.964239:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.964245:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.964262:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.034436:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971518 is committed 00002000:00010000:3.0:1789760711.034876:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17195008 -> 17199104 00010000:00010000:3.0:1789760711.034893:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034907:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034921:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034930:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.034935:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034943:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034962:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.034969:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17195008->17199103] (req 17195008->17199103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.035150:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70987us (72104us total) trans 4294971518 rc 0/0 00000100:00100000:3.0:1789760711.035161:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4228 00000100:00100000:3.0:1789760711.035171:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692234211870 time 692279448369 state 0x2 ref 0 00000100:00100000:2.0:1789760711.046082:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.046282:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164416 00000100:00100000:3.0:1789760711.046292:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164416 00000100:00100000:3.0:1789760711.046360:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692234211870 time 692279448369 state 0x0 ref 1 00000100:00100000:3.0:1789760711.046375:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692279708785 time 692279448369 state 0x1 ref 1 00000100:00100000:3.0:1789760711.046410:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692279708785 time 692361713537 state 0x0 ref 1 00000100:00100000:3.0:1789760711.046422:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.046469:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17199104, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x8807 00010000:00010000:3.0:1789760711.046478:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.046493:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.046497:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.046504:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.046519:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760711.048626:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760711.155336:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971519 is committed 00002000:00010000:3.0:1789760711.155449:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17199104 -> 17203200 00002000:00010000:3.0:1789760711.155454:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:3.0:1789760711.155457:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:3.0:1789760711.155471:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155484:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155493:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155500:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.155505:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155514:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155535:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.155542:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17199104->17203199] (req 17199104->17203199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.155640:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 109214us (109563us total) trans 4294971519 rc 0/0 00000100:00100000:3.0:1789760711.155650:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4229 00000100:00100000:3.0:1789760711.155660:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692279708785 time 692361713537 state 0x2 ref 0 00000100:00100000:2.0:1789760711.172802:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.173137:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164544 00000100:00100000:3.0:1789760711.173147:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164544 00000100:00100000:3.0:1789760711.173209:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692279708785 time 692361713537 state 0x0 ref 1 00000100:00100000:3.0:1789760711.173224:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692361973953 time 692361713537 state 0x1 ref 1 00000100:00100000:3.0:1789760711.173252:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692361973953 time 692488555323 state 0x0 ref 1 00000100:00100000:3.0:1789760711.173263:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.173307:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17203200, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.173315:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.173330:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.173334:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.173339:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.173353:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.266207:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971520 is committed 00002000:00010000:3.0:1789760711.266740:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17203200 -> 17207296 00010000:00010000:3.0:1789760711.266755:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266766:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266775:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266782:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.266787:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266795:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266814:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.266822:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17203200->17207295] (req 17203200->17207295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.266925:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 93657us (94127us total) trans 4294971520 rc 0/0 00000100:00100000:3.0:1789760711.268026:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4230 00000100:00100000:3.0:1789760711.268038:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692361973953 time 692488555323 state 0x2 ref 0 00000100:00100000:0.0:1789760711.278520:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.278685:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164672 00000100:00100000:3.0:1789760711.278725:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164672 00000100:00100000:3.0:1789760711.278794:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692361973953 time 692488555323 state 0x0 ref 1 00000100:00100000:3.0:1789760711.278808:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692488815739 time 692488555323 state 0x1 ref 1 00000100:00100000:3.0:1789760711.278842:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692488815739 time 692594144547 state 0x0 ref 1 00000100:00100000:3.0:1789760711.278854:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507164672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.278903:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17207296, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.278911:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.278924:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.278927:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.278931:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.278942:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760711.349308:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17207296 -> 17211392 00010000:00010000:3.0:1789760711.349328:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349342:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349352:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349360:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.349365:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349374:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349404:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.349413:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17207296->17211391] (req 17207296->17211391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.349524:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507164672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70665us (71006us total) trans 4294971521 rc 0/0 00000100:00100000:3.0:1789760711.349534:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4231 00000100:00100000:3.0:1789760711.349545:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692488815739 time 692594144547 state 0x2 ref 0 00000001:00080000:0.0:1789760711.350788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971521 is committed 00000100:00100000:2.0:1789760711.362165:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.362296:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164800 00000100:00100000:3.0:1789760711.362307:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164800 00000100:00100000:3.0:1789760711.362362:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692488815739 time 692594144547 state 0x0 ref 1 00000100:00100000:3.0:1789760711.362377:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692594404963 time 692594144547 state 0x1 ref 1 00000100:00100000:3.0:1789760711.362406:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692594404963 time 692677709200 state 0x0 ref 1 00000100:00100000:3.0:1789760711.362419:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.362476:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17211392, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.362485:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.363490:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.363560:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.363567:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.363718:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.430834:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971522 is committed 00002000:00010000:3.0:1789760711.431095:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17211392 -> 17215488 00010000:00010000:3.0:1789760711.431111:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431124:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431136:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431144:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.431148:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431157:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431177:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.431184:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17211392->17215487] (req 17211392->17215487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.431278:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68857us (69117us total) trans 4294971522 rc 0/0 00000100:00100000:3.0:1789760711.431288:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4232 00000100:00100000:3.0:1789760711.431297:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692594404963 time 692677709200 state 0x2 ref 0 00000100:00100000:1.0:1789760711.435810:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760711.436009:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507164928 00000100:00100000:2.0:1789760711.436028:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507164928 00000100:00100000:2.0:1789760711.436090:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692594404963 time 692677709200 state 0x0 ref 1 00000100:00100000:2.0:1789760711.436105:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692677969616 time 692677709200 state 0x1 ref 1 00000100:00100000:2.0:1789760711.436129:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692677969616 time 692751433117 state 0x0 ref 1 00000100:00100000:2.0:1789760711.436139:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760711.436179:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17215488, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:2.0:1789760711.436186:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.436202:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760711.436205:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.436209:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.436218:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.488088:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971523 is committed 00002000:00010000:3.0:1789760711.488307:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17215488 -> 17219584 00010000:00010000:3.0:1789760711.488323:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488344:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488353:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488360:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.488365:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488375:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488395:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.488404:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b6f28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17215488->17219583] (req 17215488->17219583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.489039:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507164928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52895us (53232us total) trans 4294971523 rc 0/0 00000100:00100000:3.0:1789760711.489051:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4233 00000100:00100000:3.0:1789760711.489061:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692677969616 time 692751433117 state 0x2 ref 0 00000100:00100000:0.0:1789760711.495069:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.495885:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165056 00000100:00100000:3.0:1789760711.495895:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165056 00000100:00100000:3.0:1789760711.495959:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692677969616 time 692751433117 state 0x0 ref 1 00000100:00100000:3.0:1789760711.495976:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692751693533 time 692751433117 state 0x1 ref 1 00000100:00100000:3.0:1789760711.496009:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692751693533 time 692811312727 state 0x0 ref 1 00000100:00100000:3.0:1789760711.496032:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507165056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.496092:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17219584, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.496102:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.496117:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.496121:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.496127:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.496141:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.564042:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971524 is committed 00002000:00010000:3.0:1789760711.566767:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17219584 -> 17223680 00010000:00010000:3.0:1789760711.566789:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566803:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566818:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566825:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.566829:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566838:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566860:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.566868:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17219584->17223679] (req 17219584->17223679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.566973:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507165056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70939us (71917us total) trans 4294971524 rc 0/0 00000100:00100000:3.0:1789760711.566984:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4234 00000100:00100000:3.0:1789760711.566993:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692751693533 time 692811312727 state 0x2 ref 0 00000100:00100000:2.0:1789760711.572819:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.572995:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165184 00000100:00100000:3.0:1789760711.573007:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165184 00000100:00100000:3.0:1789760711.573089:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692751693533 time 692811312727 state 0x0 ref 1 00000100:00100000:3.0:1789760711.573277:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692811573143 time 692811312727 state 0x1 ref 1 00000100:00100000:3.0:1789760711.573314:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692811573143 time 692888617262 state 0x0 ref 1 00000100:00100000:3.0:1789760711.573328:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.573379:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17223680, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.573387:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.573404:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760711.573410:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.573416:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.573430:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.585367:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760711.640967:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971525 is committed 00002000:00010000:0.0:1789760711.641251:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17223680 -> 17227776 00010000:00010000:0.0:1789760711.641266:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641278:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641287:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641294:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760711.641298:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641306:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641325:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.641333:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b6f36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17223680->17227775] (req 17223680->17227775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.641464:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68126us (68643us total) trans 4294971525 rc 0/0 00000100:00100000:0.0:1789760711.641478:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4235 00000100:00100000:0.0:1789760711.641487:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692811573143 time 692888617262 state 0x2 ref 0 00000100:00100000:1.0:1789760711.645383:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.645556:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165312 00000100:00100000:3.0:1789760711.645568:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165312 00000100:00100000:3.0:1789760711.645622:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692811573143 time 692888617262 state 0x0 ref 1 00000100:00100000:3.0:1789760711.645638:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692888877678 time 692888617262 state 0x1 ref 1 00000100:00100000:2.0:1789760711.645813:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692888877678 time 692961114028 state 0x0 ref 1 00000100:00100000:2.0:1789760711.645830:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760711.645877:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17227776, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:2.0:1789760711.645885:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645900:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760711.645904:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.645910:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645924:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.684987:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971526 is committed 00002000:00010000:2.0:1789760711.685092:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17227776 -> 17231872 00010000:00010000:2.0:1789760711.685107:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685119:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685129:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685136:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760711.685141:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685149:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685167:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.685174:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b6f3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17227776->17231871] (req 17227776->17231871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.685256:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39426us (39880us total) trans 4294971526 rc 0/0 00000100:00100000:2.0:1789760711.685266:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4236 00000100:00100000:2.0:1789760711.685276:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692888877678 time 692961114028 state 0x2 ref 0 00000100:00100000:1.0:1789760711.690282:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.690342:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165440 00000100:00100000:3.0:1789760711.690352:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165440 00000100:00100000:3.0:1789760711.690403:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692888877678 time 692961114028 state 0x0 ref 1 00000100:00100000:3.0:1789760711.690418:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692961374444 time 692961114028 state 0x1 ref 1 00000100:00100000:3.0:1789760711.690451:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692961374444 time 693005754128 state 0x0 ref 1 00000100:00100000:3.0:1789760711.690462:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.690509:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17231872, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.690651:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.690668:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760711.690672:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.690679:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.690722:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.734361:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971527 is committed 00002000:00010000:3.0:1789760711.734491:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17231872 -> 17235968 00010000:00010000:3.0:1789760711.734508:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734521:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734531:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734539:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.734545:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734554:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734574:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.734583:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17231872->17235967] (req 17231872->17235967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.734677:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44211us (44398us total) trans 4294971527 rc 0/0 00000100:00100000:3.0:1789760711.734688:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4237 00000100:00100000:3.0:1789760711.734725:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692961374444 time 693005754128 state 0x2 ref 0 00000100:00100000:2.0:1789760711.740177:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760711.740272:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165568 00000100:00100000:0.0:1789760711.740283:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165568 00000100:00100000:0.0:1789760711.740340:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 692961374444 time 693005754128 state 0x0 ref 1 00000100:00100000:0.0:1789760711.740357:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693006014544 time 693005754128 state 0x1 ref 1 00000100:00100000:0.0:1789760711.740424:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693006014544 time 693055726541 state 0x0 ref 1 00000100:00100000:0.0:1789760711.740438:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760711.740554:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17235968, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:0.0:1789760711.740563:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.740580:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760711.740584:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.740591:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.740606:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.783685:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971528 is committed 00002000:00010000:3.0:1789760711.785756:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17235968 -> 17240064 00010000:00010000:3.0:1789760711.785777:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785790:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785798:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785811:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760711.785817:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785826:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785854:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.785862:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6f4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17235968->17240063] (req 17235968->17240063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.785968:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45526us (45914us total) trans 4294971528 rc 0/0 00000100:00100000:3.0:1789760711.785978:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4238 00000100:00100000:3.0:1789760711.785989:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693006014544 time 693055726541 state 0x2 ref 0 00000100:00100000:2.0:1789760711.792761:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760711.792848:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165696 00000100:00100000:0.0:1789760711.792858:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165696 00000100:00100000:0.0:1789760711.792911:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693006014544 time 693055726541 state 0x0 ref 1 00000100:00100000:0.0:1789760711.792926:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693055986957 time 693055726541 state 0x1 ref 1 00000100:00100000:0.0:1789760711.792963:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693055986957 time 693108266489 state 0x0 ref 1 00000100:00100000:0.0:1789760711.792975:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507165696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760711.793031:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17240064, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:0.0:1789760711.793041:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.793235:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760711.793239:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.793246:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.793262:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.834530:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971529 is committed 00002000:00010000:0.0:1789760711.835412:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17240064 -> 17244160 00010000:00010000:0.0:1789760711.835429:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835440:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835450:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835469:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760711.835474:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835483:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835503:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.835511:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17240064->17244159] (req 17240064->17244159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.835644:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507165696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42665us (42888us total) trans 4294971529 rc 0/0 00000100:00100000:0.0:1789760711.835655:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4239 00000100:00100000:0.0:1789760711.835665:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693055986957 time 693108266489 state 0x2 ref 0 00000100:00100000:1.0:1789760711.841454:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760711.842589:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165824 00000100:00100000:3.0:1789760711.842599:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165824 00000100:00100000:3.0:1789760711.842647:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693055986957 time 693108266489 state 0x0 ref 1 00000100:00100000:3.0:1789760711.842659:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693108526905 time 693108266489 state 0x1 ref 1 00000100:00100000:3.0:1789760711.842730:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693108526905 time 693158031391 state 0x0 ref 1 00000100:00100000:3.0:1789760711.842747:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760711.842797:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17244160, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:3.0:1789760711.842805:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.842823:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760711.842828:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.842833:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.842848:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.894852:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971530 is committed 00002000:00010000:1.0:1789760711.895029:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17244160 -> 17248256 00010000:00010000:1.0:1789760711.895103:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895117:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895126:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895133:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760711.895139:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895148:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895173:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.895181:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b6f59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17244160->17248255] (req 17244160->17248255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.895277:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52528us (53828us total) trans 4294971530 rc 0/0 00000100:00100000:1.0:1789760711.895287:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4240 00000100:00100000:1.0:1789760711.895297:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693108526905 time 693158031391 state 0x2 ref 0 00000100:00100000:3.0:1789760711.899248:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760711.901688:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507165952 00000100:00100000:0.0:1789760711.901716:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507165952 00000100:00100000:0.0:1789760711.901775:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693108526905 time 693158031391 state 0x0 ref 1 00000100:00100000:0.0:1789760711.901792:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693158291807 time 693158031391 state 0x1 ref 1 00000100:00100000:0.0:1789760711.901826:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693158291807 time 693217129038 state 0x0 ref 1 00000100:00100000:0.0:1789760711.901842:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760711.901890:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17248256, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:0.0:1789760711.901898:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.901914:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760711.901919:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.901926:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.901941:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.937232:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971531 is committed 00002000:00010000:1.0:1789760711.937331:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17248256 -> 17252352 00010000:00010000:1.0:1789760711.937346:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937359:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937367:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937374:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760711.937379:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937393:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937411:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.937419:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6f60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17248256->17252351] (req 17248256->17252351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.937633:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507165952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35787us (38389us total) trans 4294971531 rc 0/0 00000100:00100000:1.0:1789760711.937644:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4241 00000100:00100000:1.0:1789760711.937654:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693158291807 time 693217129038 state 0x2 ref 0 00000100:00100000:2.0:1789760711.941953:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760711.942187:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166080 00000100:00100000:0.0:1789760711.942198:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166080 00000100:00100000:0.0:1789760711.942249:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693158291807 time 693217129038 state 0x0 ref 1 00000100:00100000:0.0:1789760711.942264:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693217389454 time 693217129038 state 0x1 ref 1 00000100:00100000:0.0:1789760711.942332:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693217389454 time 693257634604 state 0x0 ref 1 00000100:00100000:0.0:1789760711.942346:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507166080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760711.942391:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17252352, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:0.0:1789760711.942400:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.942420:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760711.942425:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.942431:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.942444:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.996506:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17252352 -> 17256448 00010000:00010000:1.0:1789760711.996526:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996538:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996547:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996553:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760711.996559:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996569:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996589:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.996597:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17252352->17256447] (req 17252352->17256447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.996724:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507166080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54343us (54744us total) trans 4294971532 rc 0/0 00000100:00100000:1.0:1789760711.996736:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4242 00000001:00080000:2.0:1789760711.996738:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971532 is committed 00000100:00100000:1.0:1789760711.996748:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693217389454 time 693257634604 state 0x2 ref 0 00000100:00100000:2.0:1789760712.001340:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.001413:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166208 00000100:00100000:1.0:1789760712.001423:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166208 00000100:00100000:1.0:1789760712.001489:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693217389454 time 693257634604 state 0x0 ref 1 00000100:00100000:1.0:1789760712.001503:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693257895020 time 693257634604 state 0x1 ref 1 00000100:00100000:1.0:1789760712.001532:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693257895020 time 693316836245 state 0x0 ref 1 00000100:00100000:1.0:1789760712.001545:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507166208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760712.001595:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17256448, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x8807 00010000:00010000:1.0:1789760712.001604:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.001621:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760712.001625:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.001631:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.001644:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.064073:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971533 is committed 00002000:00010000:3.0:1789760712.064167:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17256448 -> 17260544 00002000:00010000:3.0:1789760712.064172:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:3.0:1789760712.064175:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:3.0:1789760712.064200:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064214:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064224:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064239:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760712.064244:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064255:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064277:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.064285:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b6f6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17256448->17260543] (req 17256448->17260543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.064373:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507166208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62826us (63035us total) trans 4294971533 rc 0/0 00000100:00100000:3.0:1789760712.064393:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4243 00000100:00100000:3.0:1789760712.064403:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693257895020 time 693316836245 state 0x2 ref 0 00000100:00100000:3.0:1789760712.069849:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.071858:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166336 00000100:00100000:1.0:1789760712.071874:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166336 00000100:00100000:1.0:1789760712.071938:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693257895020 time 693316836245 state 0x0 ref 1 00000100:00100000:1.0:1789760712.071950:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693317096661 time 693316836245 state 0x1 ref 1 00000100:00100000:1.0:1789760712.072114:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693317096661 time 693387415534 state 0x0 ref 1 00000100:00100000:1.0:1789760712.072128:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240b8c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760712.072177:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17260544, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:1.0:1789760712.072185:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.072198:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760712.072202:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.072209:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.072223:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.092823:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971534 is committed 00002000:00010000:1.0:1789760712.092925:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17260544 -> 17264640 00010000:00010000:1.0:1789760712.092940:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.092951:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.092961:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.092968:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.092973:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.092982:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.093001:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.093020:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17260544->17264639] (req 17260544->17264639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.093217:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240b8c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21087us (23373us total) trans 4294971534 rc 0/0 00000100:00100000:1.0:1789760712.093228:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4244 00000100:00100000:1.0:1789760712.093237:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693317096661 time 693387415534 state 0x2 ref 0 00000100:00100000:2.0:1789760712.103465:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.103542:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166464 00000100:00100000:1.0:1789760712.103551:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166464 00000100:00100000:1.0:1789760712.103599:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693317096661 time 693387415534 state 0x0 ref 1 00000100:00100000:1.0:1789760712.103611:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693387675950 time 693387415534 state 0x1 ref 1 00000100:00100000:1.0:1789760712.103656:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693387675950 time 693418960113 state 0x0 ref 1 00000100:00100000:1.0:1789760712.103666:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760712.103788:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17264640, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:1.0:1789760712.103798:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.103813:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760712.103818:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.103825:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.103839:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.153395:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971535 is committed 00002000:00010000:0.0:1789760712.153506:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17264640 -> 17268736 00010000:00010000:0.0:1789760712.153521:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153534:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153543:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153550:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760712.153554:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153564:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153582:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.153591:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b6f7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17264640->17268735] (req 17264640->17268735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.153897:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50227us (50444us total) trans 4294971535 rc 0/0 00000100:00100000:0.0:1789760712.153907:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4245 00000100:00100000:0.0:1789760712.153917:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693387675950 time 693418960113 state 0x2 ref 0 00000100:00100000:2.0:1789760712.160645:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.160909:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166592 00000100:00100000:3.0:1789760712.160918:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166592 00000100:00100000:3.0:1789760712.160973:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693387675950 time 693418960113 state 0x0 ref 1 00000100:00100000:3.0:1789760712.160987:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693419220529 time 693418960113 state 0x1 ref 1 00000100:00100000:3.0:1789760712.161068:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693419220529 time 693476370237 state 0x0 ref 1 00000100:00100000:3.0:1789760712.161082:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507166592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760712.161127:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17268736, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:3.0:1789760712.161137:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.161153:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760712.161157:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.161164:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.161178:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.226588:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17268736 -> 17272832 00010000:00010000:3.0:1789760712.226608:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226620:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226628:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226634:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760712.226639:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226646:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226666:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.226674:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b6f83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17268736->17272831] (req 17268736->17272831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.226807:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507166592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65722us (66166us total) trans 4294971536 rc 0/0 00000100:00100000:3.0:1789760712.226818:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4246 00000100:00100000:3.0:1789760712.226829:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693419220529 time 693476370237 state 0x2 ref 0 00000001:00080000:3.0:1789760712.226869:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971536 is committed 00000100:00100000:2.0:1789760712.231902:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.235880:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166720 00000100:00100000:1.0:1789760712.235897:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166720 00000100:00100000:1.0:1789760712.235962:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693419220529 time 693476370237 state 0x0 ref 1 00000100:00100000:1.0:1789760712.235977:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693476630653 time 693476370237 state 0x1 ref 1 00000100:00100000:1.0:1789760712.236022:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693476630653 time 693551324746 state 0x0 ref 1 00000100:00100000:1.0:1789760712.236036:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760712.236093:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17272832, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:1.0:1789760712.236102:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.236118:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760712.236123:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.236128:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.236144:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.367528:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971537 is committed 00002000:00010000:2.0:1789760712.367761:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17272832 -> 17276928 00010000:00010000:2.0:1789760712.367777:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367790:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367800:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367807:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760712.367812:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367825:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367847:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.367854:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b6f8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17272832->17276927] (req 17272832->17276927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.367951:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507166720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 131914us (136054us total) trans 4294971537 rc 0/0 00000100:00100000:2.0:1789760712.367961:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4247 00000100:00100000:2.0:1789760712.367971:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693476630653 time 693551324746 state 0x2 ref 0 00000100:00100000:2.0:1789760712.372481:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.372614:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166848 00000100:00100000:0.0:1789760712.372625:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166848 00000100:00100000:0.0:1789760712.372683:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693476630653 time 693551324746 state 0x0 ref 1 00000100:00100000:0.0:1789760712.372728:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693551585162 time 693551324746 state 0x1 ref 1 00000100:00100000:0.0:1789760712.372772:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693551585162 time 693688075593 state 0x0 ref 1 00000100:00100000:0.0:1789760712.372786:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507166848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.372838:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17276928, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.372847:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.372862:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760712.372867:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.372873:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.372891:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.406302:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.406335:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.406378:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.406397:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.406452:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760712.406480:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167104 00000100:00100000:3.0:1789760712.406489:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167104 00000100:00100000:3.0:1789760712.406543:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 828 00000100:00100000:3.0:1789760712.406555:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507167104:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760712.406627:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507167104:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 74us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.406634:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 828 00000100:00100000:3.0:1789760712.406653:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167360 00000100:00100000:3.0:1789760712.406657:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167360 00000100:00100000:3.0:1789760712.406671:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167232 00000100:00100000:3.0:1789760712.406673:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167232 00000100:00100000:3.0:1789760712.406684:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167488 00000100:00100000:3.0:1789760712.406686:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167488 00000100:00100000:3.0:1789760712.407125:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 829 00000100:00100000:3.0:1789760712.407138:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507167360:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760712.407170:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507167360:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 37us (834us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.407177:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 829 00000100:00100000:3.0:1789760712.407191:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 830 00000100:00100000:3.0:1789760712.407197:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507167232:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760712.407221:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507167232:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 24us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.407227:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 830 00000100:00100000:3.0:1789760712.407239:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 831 00000100:00100000:3.0:1789760712.407245:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507167488:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760712.407265:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507167488:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 20us (813us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.407270:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 831 00000100:00100000:3.0:1789760712.407293:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507166976 00000100:00100000:3.0:1789760712.407298:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507166976 00000100:00100000:3.0:1789760712.407326:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 456 00000100:00100000:3.0:1789760712.407332:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223496850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507166976:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760712.407420:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223496850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507166976:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 87us (1041us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.407427:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 456 00000100:00080000:3.0:1789760712.407459:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760712.407463:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:3.0:1789760712.469609:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17276928 -> 17281024 00010000:00010000:3.0:1789760712.469631:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469644:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469654:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469661:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760712.469666:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469675:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469711:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.469725:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b6f91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17276928->17281023] (req 17276928->17281023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.469958:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507166848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 97170us (97482us total) trans 4294971538 rc 0/0 00000100:00100000:3.0:1789760712.469967:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4248 00000100:00100000:3.0:1789760712.469977:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693551585162 time 693688075593 state 0x2 ref 0 00000001:00080000:3.0:1789760712.470036:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971538 is committed 00000100:00100000:3.0:1789760712.472950:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.479601:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167616 00000100:00100000:1.0:1789760712.479617:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167616 00000100:00100000:0.0:1789760712.479794:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693551585162 time 693688075593 state 0x0 ref 1 00000100:00100000:0.0:1789760712.479809:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693688336009 time 693688075593 state 0x1 ref 1 00000100:00100000:0.0:1789760712.479843:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693688336009 time 693795146667 state 0x0 ref 1 00000100:00100000:0.0:1789760712.479856:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507167616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.479902:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17281024, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.479912:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.479928:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760712.479932:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.479938:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.479953:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.585082:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971539 is committed 00002000:00010000:0.0:1789760712.585246:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17281024 -> 17285120 00010000:00010000:0.0:1789760712.585262:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585276:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585286:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585295:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760712.585299:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585308:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585329:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.585336:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6f98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17281024->17285119] (req 17281024->17285119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.585443:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507167616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 105584us (112495us total) trans 4294971539 rc 0/0 00000100:00100000:0.0:1789760712.585454:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4249 00000100:00100000:0.0:1789760712.585463:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693688336009 time 693795146667 state 0x2 ref 0 00000100:00100000:3.0:1789760712.589688:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760712.590181:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167744 00000100:00100000:1.0:1789760712.590191:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167744 00000100:00100000:1.0:1789760712.590245:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693688336009 time 693795146667 state 0x0 ref 1 00000100:00100000:1.0:1789760712.590261:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693795407083 time 693795146667 state 0x1 ref 1 00000100:00100000:1.0:1789760712.590291:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693795407083 time 693905594730 state 0x0 ref 1 00000100:00100000:1.0:1789760712.590304:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507167744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760712.590361:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17285120, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:1.0:1789760712.590370:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.590386:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760712.590390:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.590397:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.590410:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.604544:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760712.629289:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17285120 -> 17289216 00010000:00010000:2.0:1789760712.629308:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629321:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629330:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629343:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760712.629348:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629366:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629385:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.629392:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b6f9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17285120->17289215] (req 17285120->17289215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.629612:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507167744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39304us (39928us total) trans 4294971540 rc 0/0 00000100:00100000:2.0:1789760712.629622:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4250 00000100:00100000:2.0:1789760712.629633:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693795407083 time 693905594730 state 0x2 ref 0 00000001:00080000:0.0:1789760712.632138:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971540 is committed 00000100:00100000:3.0:1789760712.634562:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.634648:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507167872 00000100:00100000:0.0:1789760712.634658:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507167872 00000100:00100000:0.0:1789760712.634867:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693795407083 time 693905594730 state 0x0 ref 1 00000100:00100000:0.0:1789760712.634882:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693905855146 time 693905594730 state 0x1 ref 1 00000100:00100000:0.0:1789760712.634949:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693905855146 time 693950251636 state 0x0 ref 1 00000100:00100000:0.0:1789760712.634962:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507167872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.635018:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17289216, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.635027:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.635051:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760712.635055:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.635062:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.635077:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.667146:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971541 is committed 00002000:00010000:1.0:1789760712.667229:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17289216 -> 17293312 00010000:00010000:1.0:1789760712.667244:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667256:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667264:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667277:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.667281:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667290:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667310:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.667317:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b6fa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17289216->17293311] (req 17289216->17293311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.667577:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507167872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32612us (33019us total) trans 4294971541 rc 0/0 00000100:00100000:1.0:1789760712.667587:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4251 00000100:00100000:1.0:1789760712.667597:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693905855146 time 693950251636 state 0x2 ref 0 00000100:00100000:3.0:1789760712.673361:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.673432:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168000 00000100:00100000:0.0:1789760712.673443:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168000 00000100:00100000:0.0:1789760712.673513:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693905855146 time 693950251636 state 0x0 ref 1 00000100:00100000:0.0:1789760712.673528:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693950512052 time 693950251636 state 0x1 ref 1 00000100:00100000:0.0:1789760712.673560:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693950512052 time 693988863217 state 0x0 ref 1 00000100:00100000:0.0:1789760712.673573:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.673622:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17293312, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.673631:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.673647:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760712.673652:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.673658:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.673674:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.732366:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971542 is committed 00002000:00010000:1.0:1789760712.732783:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17293312 -> 17297408 00010000:00010000:1.0:1789760712.732800:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732812:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732827:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732834:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.732838:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732847:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732869:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732949:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b6fad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17293312->17297407] (req 17293312->17297407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.733112:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59536us (59768us total) trans 4294971542 rc 0/0 00000100:00100000:1.0:1789760712.733123:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4252 00000100:00100000:1.0:1789760712.733133:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693950512052 time 693988863217 state 0x2 ref 0 00000100:00100000:3.0:1789760712.737435:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760712.738229:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168128 00000100:00100000:2.0:1789760712.738239:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168128 00000100:00100000:2.0:1789760712.738291:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693950512052 time 693988863217 state 0x0 ref 1 00000100:00100000:2.0:1789760712.738306:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693989123633 time 693988863217 state 0x1 ref 1 00000100:00100000:2.0:1789760712.738333:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693989123633 time 694053636953 state 0x0 ref 1 00000100:00100000:2.0:1789760712.738360:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760712.738410:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17297408, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:2.0:1789760712.738419:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738434:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760712.738439:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.738445:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738561:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.770997:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17297408 -> 17301504 00010000:00010000:1.0:1789760712.771608:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771628:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771638:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771646:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.771651:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771660:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771682:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.771690:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b6fb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17297408->17301503] (req 17297408->17301503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.771878:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971543 is committed 00000100:00100000:2.0:1789760712.773233:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34879us (35803us total) trans 4294971543 rc 0/0 00000100:00100000:2.0:1789760712.773244:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4253 00000100:00100000:2.0:1789760712.773253:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693989123633 time 694053636953 state 0x2 ref 0 00000100:00100000:1.0:1789760712.781026:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.781168:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168256 00000100:00100000:0.0:1789760712.781179:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168256 00000100:00100000:0.0:1789760712.781230:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 693989123633 time 694053636953 state 0x0 ref 1 00000100:00100000:0.0:1789760712.781246:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694053897369 time 694053636953 state 0x1 ref 1 00000100:00100000:0.0:1789760712.781274:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694053897369 time 694096577183 state 0x0 ref 1 00000100:00100000:0.0:1789760712.781285:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.781331:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17301504, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.781339:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.781354:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760712.781359:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.781365:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.781379:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.849335:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17301504 -> 17305600 00010000:00010000:1.0:1789760712.849356:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849369:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849437:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849509:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.849515:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849525:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849543:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.849551:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b6fbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17301504->17305599] (req 17301504->17305599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.849651:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507168256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68361us (68629us total) trans 4294971544 rc 0/0 00000100:00100000:1.0:1789760712.849661:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4254 00000100:00100000:1.0:1789760712.849670:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694053897369 time 694096577183 state 0x2 ref 0 00000001:00080000:0.0:1789760712.852751:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971544 is committed 00000100:00100000:1.0:1789760712.854265:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.854556:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168384 00000100:00100000:0.0:1789760712.854565:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168384 00000100:00100000:0.0:1789760712.854617:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694053897369 time 694096577183 state 0x0 ref 1 00000100:00100000:0.0:1789760712.854632:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694096837599 time 694096577183 state 0x1 ref 1 00000100:00100000:0.0:1789760712.854662:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694096837599 time 694169964904 state 0x0 ref 1 00000100:00100000:0.0:1789760712.854673:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760712.854753:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17305600, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:0.0:1789760712.854935:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.854952:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760712.854956:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.854962:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.854977:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.924829:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538259712:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760712.924896:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760712.925173:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259712 00000100:00100000:0.0:1789760712.925182:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259712 00000100:00100000:0.0:1789760712.925241:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:0.0:1789760712.925251:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538259712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760712.925285:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760712.925332:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538259712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760712.925338:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:3.0:1789760712.927681:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538259968:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.927991:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.928114:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538260096:0@lo:400:kworker.0 00000100:00080000:1.0:1789760712.928179:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760712.928203:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538260224:0@lo:400:kworker.0 00000100:00100000:0.0:1789760712.930590:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259968 00000100:00100000:0.0:1789760712.930604:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259968 00000100:00100000:0.0:1789760712.930661:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:0.0:1789760712.930670:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223494c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538259968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.930961:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.931029:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538260608:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931064:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.931088:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538261120:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931108:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.931148:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260096 00000100:00100000:2.0:1789760712.931151:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260096 00000100:00100000:2.0:1789760712.931177:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260608 00000100:00100000:2.0:1789760712.931193:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260608 00000100:00100000:2.0:1789760712.931223:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261120 00000100:00100000:2.0:1789760712.931226:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261120 00000100:00100000:2.0:1789760712.931241:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 832 00000100:00100000:2.0:1789760712.931248:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538260096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931288:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538260096:12345-0@lo:400:kworker.0 Request processed in 40us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760712.931295:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 832 00000100:00100000:2.0:1789760712.931312:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 833 00000100:00100000:2.0:1789760712.931318:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538260608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931345:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538260608:12345-0@lo:400:kworker.0 Request processed in 28us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760712.931351:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 833 00000100:00100000:2.0:1789760712.931361:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 834 00000100:00100000:2.0:1789760712.931366:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538261120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931402:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538261120:12345-0@lo:400:kworker.0 Request processed in 36us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760712.931407:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 834 00000100:00080000:2.0:1789760712.931429:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760712.931472:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538260096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931483:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538260608:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.931492:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538261120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760712.932364:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760712.932411:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538260736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760712.932452:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760712.932472:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538261248:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.932496:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260736 00000100:00100000:1.0:1789760712.932497:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.932505:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260736 00000100:00100000:1.0:1789760712.932539:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538259712:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760712.932550:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261248 00000100:00100000:2.0:1789760712.932555:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261248 00000100:00100000:1.0:1789760712.932562:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1789760712.932571:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538259840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760712.932581:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 835 00000100:00100000:2.0:1789760712.932591:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538260736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760712.932602:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760712.932626:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538260352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.932652:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538260736:12345-0@lo:400:kworker.0 Request processed in 61us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760712.932663:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.932663:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 835 00000100:00100000:2.0:1789760712.932681:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260352 00000100:00100000:2.0:1789760712.932686:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260352 00000100:00100000:1.0:1789760712.932690:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538260864:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.932731:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 836 00000100:00100000:2.0:1789760712.932739:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538261248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760712.932755:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.932778:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538261248:12345-0@lo:400:kworker.0 Request processed in 38us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760712.932785:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 836 00000100:00100000:1.0:1789760712.932789:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538261376:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.932801:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 837 00000100:00100000:2.0:1789760712.932809:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538260352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760712.932818:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760712.932854:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538260352:12345-0@lo:400:kworker.0 Request processed in 45us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760712.932855:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538260736:0@lo:400:kworker.0 00000100:00100000:2.0:1789760712.932865:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1789760712.932874:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538261248:0@lo:400:kworker.0 00000100:00080000:2.0:1789760712.932890:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760712.932898:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538260352:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.933666:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260224 00000100:00100000:3.0:1789760712.933675:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260224 00000100:00100000:3.0:1789760712.933739:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789760712.933748:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538260224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.933918:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538260224:12345-0@lo:400:kworker.0 Request processed in 169us (1554us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.933926:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789760712.933961:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538259840 00000100:00100000:3.0:1789760712.933965:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538259840 00000100:00100000:3.0:1789760712.934658:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:3.0:1789760712.934668:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538259840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760712.934686:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760712.935886:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538259840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1216us (3280us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.935896:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:3.0:1789760712.935933:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260864 00000100:00100000:3.0:1789760712.935939:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260864 00000100:00100000:3.0:1789760712.935986:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261376 00000100:00100000:3.0:1789760712.935989:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261376 00000100:00100000:3.0:1789760712.936083:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789760712.936092:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538260864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.936130:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538260864:12345-0@lo:400:kworker.0 Request processed in 41us (3377us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.936137:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789760712.936149:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789760712.936156:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538261376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.936183:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538261376:12345-0@lo:400:kworker.0 Request processed in 28us (3365us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.936188:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789760712.936241:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105821f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538259840:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760712.936254:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538260864:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.936263:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538261376:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.936274:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538260224:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.936341:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538260480:0@lo:400:kworker.0 00000100:00080000:2.0:1789760712.936442:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760712.936447:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760712.937716:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760712.937880:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538260992:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.937926:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760712.937975:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260992 00000100:00100000:3.0:1789760712.937982:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260992 00000100:00100000:1.0:1789760712.938302:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538260480 00000100:00100000:1.0:1789760712.938307:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538260480 00000100:00100000:1.0:1789760712.938347:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760712.938356:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538260480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.938605:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1789760712.938612:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538260992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.938662:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538260992:12345-0@lo:400:kworker.0 Request processed in 50us (734us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.938668:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1789760712.938890:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538260480:12345-0@lo:400:kworker.0 Request processed in 534us (2511us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760712.938897:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:3.0:1789760712.938951:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538260992:0@lo:400:kworker.0 00000100:00100000:3.0:1789760712.938977:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538260480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760712.940503:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223494c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538259968:12345-0@lo:400:kworker.0 Request processed in 9829us (12506us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760712.940516:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1789760712.940953:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538259968:0@lo:400:kworker.0 00000100:00080000:2.0:1789760712.945433:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760712.948729:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971545 is committed 00002000:00010000:1.0:1789760712.948986:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17305600 -> 17309696 00010000:00010000:1.0:1789760712.949021:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949034:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949044:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949052:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760712.949056:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949065:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949087:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.949094:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b6fc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17305600->17309695] (req 17305600->17309695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.949190:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 94513us (94926us total) trans 4294971545 rc 0/0 00000100:00100000:1.0:1789760712.949202:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4255 00000100:00100000:1.0:1789760712.949211:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694096837599 time 694169964904 state 0x2 ref 0 00000100:00100000:2.0:1789760712.953041:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760712.954170:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168512 00000100:00100000:0.0:1789760712.954181:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168512 00000100:00100000:0.0:1789760712.954240:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694096837599 time 694169964904 state 0x0 ref 1 00000100:00100000:0.0:1789760712.954255:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694170225320 time 694169964904 state 0x1 ref 1 00000100:00100000:0.0:1789760712.954287:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694170225320 time 694269589744 state 0x0 ref 1 00000100:00100000:3.0:1789760712.954894:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760712.954953:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17309696, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:3.0:1789760712.954964:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.954980:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760712.954984:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.954991:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.955023:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.016337:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971546 is committed 00002000:00010000:2.0:1789760713.016555:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17309696 -> 17313792 00010000:00010000:2.0:1789760713.016572:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016585:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016593:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016599:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760713.016603:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016611:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016631:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.016638:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b6fc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17309696->17313791] (req 17309696->17313791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.016810:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62129us (63772us total) trans 4294971546 rc 0/0 00000100:00100000:2.0:1789760713.016821:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4256 00000100:00100000:2.0:1789760713.016830:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694170225320 time 694269589744 state 0x2 ref 0 00000100:00100000:3.0:1789760713.027953:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.028032:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168640 00000100:00100000:1.0:1789760713.028054:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168640 00000100:00100000:1.0:1789760713.028129:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694170225320 time 694269589744 state 0x0 ref 1 00000100:00100000:1.0:1789760713.028145:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694269850160 time 694269589744 state 0x1 ref 1 00000100:00100000:1.0:1789760713.028175:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694269850160 time 694343478818 state 0x0 ref 1 00000100:00100000:1.0:1789760713.028195:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507168640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.028242:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17313792, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x8807 00010000:00010000:1.0:1789760713.028251:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.028265:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760713.028270:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.028277:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.028291:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760713.029027:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760713.113597:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971547 is committed 00002000:00010000:1.0:1789760713.113681:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17313792 -> 17317888 00002000:00010000:1.0:1789760713.113686:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:1.0:1789760713.113689:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:1.0:1789760713.113833:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113846:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113855:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113863:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.113868:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113876:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113896:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.113904:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b6fd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17313792->17317887] (req 17313792->17317887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.114044:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507168640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 85839us (86089us total) trans 4294971547 rc 0/0 00000100:00100000:1.0:1789760713.114056:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4257 00000100:00100000:1.0:1789760713.114066:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694269850160 time 694343478818 state 0x2 ref 0 00000100:00100000:1.0:1789760713.119393:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.120058:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168768 00000100:00100000:2.0:1789760713.120069:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168768 00000100:00100000:2.0:1789760713.120128:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694269850160 time 694343478818 state 0x0 ref 1 00000100:00100000:2.0:1789760713.120143:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694343739234 time 694343478818 state 0x1 ref 1 00000100:00100000:2.0:1789760713.120177:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694343739234 time 694435480126 state 0x0 ref 1 00000100:00100000:2.0:1789760713.120190:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.120239:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17317888, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.120248:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.120264:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.120269:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.120275:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.120290:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.170370:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971548 is committed 00002000:00010000:2.0:1789760713.170768:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17317888 -> 17321984 00010000:00010000:2.0:1789760713.170785:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170798:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170808:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170816:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760713.170821:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170831:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170851:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.170859:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17317888->17321983] (req 17317888->17321983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.170951:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50757us (51572us total) trans 4294971548 rc 0/0 00000100:00100000:2.0:1789760713.170961:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4258 00000100:00100000:2.0:1789760713.170971:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694343739234 time 694435480126 state 0x2 ref 0 00000100:00100000:2.0:1789760713.178566:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760713.178655:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507168896 00000100:00100000:3.0:1789760713.178666:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507168896 00000100:00100000:3.0:1789760713.178745:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694343739234 time 694435480126 state 0x0 ref 1 00000100:00100000:3.0:1789760713.178770:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694435740542 time 694435480126 state 0x1 ref 1 00000100:00100000:3.0:1789760713.178802:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694435740542 time 694494105030 state 0x0 ref 1 00000100:00100000:3.0:1789760713.178814:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760713.178861:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17321984, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:3.0:1789760713.178870:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.178887:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760713.178891:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.178897:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.178912:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.221174:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971549 is committed 00002000:00010000:0.0:1789760713.221284:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17321984 -> 17326080 00010000:00010000:0.0:1789760713.221299:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221311:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221320:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221326:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760713.221331:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221338:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221357:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.221364:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6fde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17321984->17326079] (req 17321984->17326079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.221545:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507168896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42727us (42983us total) trans 4294971549 rc 0/0 00000100:00100000:0.0:1789760713.221559:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4259 00000100:00100000:0.0:1789760713.221570:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694435740542 time 694494105030 state 0x2 ref 0 00000100:00100000:1.0:1789760713.224807:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.224889:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169024 00000100:00100000:2.0:1789760713.224902:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169024 00000100:00100000:2.0:1789760713.224955:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694435740542 time 694494105030 state 0x0 ref 1 00000100:00100000:2.0:1789760713.224975:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694494365446 time 694494105030 state 0x1 ref 1 00000100:00100000:2.0:1789760713.225015:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694494365446 time 694540318200 state 0x0 ref 1 00000100:00100000:2.0:1789760713.225028:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.225083:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17326080, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.225093:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.225108:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.225112:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.225118:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.225130:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.275788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971550 is committed 00002000:00010000:3.0:1789760713.275953:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17326080 -> 17330176 00010000:00010000:3.0:1789760713.275968:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.275980:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.275988:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.275995:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.276009:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.276019:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.276046:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.276054:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b6fe5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17326080->17330175] (req 17326080->17330175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.276145:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51115us (51344us total) trans 4294971550 rc 0/0 00000100:00100000:3.0:1789760713.276158:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4260 00000100:00100000:3.0:1789760713.276167:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694494365446 time 694540318200 state 0x2 ref 0 00000100:00100000:1.0:1789760713.280936:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.281187:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169152 00000100:00100000:2.0:1789760713.281197:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169152 00000100:00100000:2.0:1789760713.281254:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694494365446 time 694540318200 state 0x0 ref 1 00000100:00100000:2.0:1789760713.281269:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694540578616 time 694540318200 state 0x1 ref 1 00000100:00100000:2.0:1789760713.281300:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694540578616 time 694596603083 state 0x0 ref 1 00000100:00100000:2.0:1789760713.281312:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.281361:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17330176, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.281370:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.281394:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.281400:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.281407:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.281421:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.333985:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17330176 -> 17334272 00010000:00010000:3.0:1789760713.334024:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334044:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334051:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334059:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.334063:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334071:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334090:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.334098:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b6fec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17330176->17334271] (req 17330176->17334271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.334202:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507169152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52887us (53271us total) trans 4294971551 rc 0/0 00000100:00100000:3.0:1789760713.334212:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4261 00000100:00100000:3.0:1789760713.334222:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694540578616 time 694596603083 state 0x2 ref 0 00000100:00100000:3.0:1789760713.338030:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.338156:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169280 00000100:00100000:1.0:1789760713.338165:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169280 00000100:00100000:1.0:1789760713.338214:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694540578616 time 694596603083 state 0x0 ref 1 00000100:00100000:1.0:1789760713.338228:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694596863499 time 694596603083 state 0x1 ref 1 00000100:00100000:1.0:1789760713.338254:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694596863499 time 694653557993 state 0x0 ref 1 00000100:00100000:1.0:1789760713.338264:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507169280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.338305:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17334272, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.338314:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.338328:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760713.338333:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.338338:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.338350:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.340392:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971551 is committed 00000001:00080000:3.0:1789760713.380739:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971552 is committed 00002000:00010000:1.0:1789760713.380891:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17334272 -> 17338368 00010000:00010000:1.0:1789760713.380906:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380919:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380928:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380935:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.380940:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380949:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380969:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.380976:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b6ff3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17334272->17338367] (req 17334272->17338367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.381264:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42995us (43236us total) trans 4294971552 rc 0/0 00000100:00100000:1.0:1789760713.381279:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4262 00000100:00100000:1.0:1789760713.381289:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694596863499 time 694653557993 state 0x2 ref 0 00000100:00100000:1.0:1789760713.386266:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760713.386389:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169408 00000100:00100000:3.0:1789760713.386399:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169408 00000100:00100000:3.0:1789760713.386465:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694596863499 time 694653557993 state 0x0 ref 1 00000100:00100000:3.0:1789760713.386482:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694653818409 time 694653557993 state 0x1 ref 1 00000100:00100000:3.0:1789760713.386509:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694653818409 time 694701812682 state 0x0 ref 1 00000100:00100000:3.0:1789760713.386521:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760713.386573:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17338368, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:3.0:1789760713.386582:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.386597:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760713.386602:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.386608:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.386627:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.430899:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971553 is committed 00002000:00010000:3.0:1789760713.432211:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17338368 -> 17342464 00010000:00010000:3.0:1789760713.432228:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432239:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432248:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432254:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.432259:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432267:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432285:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.432292:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b6ffa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17338368->17342463] (req 17338368->17342463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.432397:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45871us (46135us total) trans 4294971553 rc 0/0 00000100:00100000:3.0:1789760713.432406:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4263 00000100:00100000:3.0:1789760713.432416:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694653818409 time 694701812682 state 0x2 ref 0 00000100:00100000:1.0:1789760713.437988:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760713.438189:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169536 00000100:00100000:3.0:1789760713.438198:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169536 00000100:00100000:3.0:1789760713.438249:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694653818409 time 694701812682 state 0x0 ref 1 00000100:00100000:3.0:1789760713.438266:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694702073098 time 694701812682 state 0x1 ref 1 00000100:00100000:3.0:1789760713.438338:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694702073098 time 694753640793 state 0x0 ref 1 00000100:00100000:3.0:1789760713.438350:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507169536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760713.438408:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17342464, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:3.0:1789760713.438417:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.438432:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760713.438436:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.438450:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.438463:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.481291:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971554 is committed 00002000:00010000:3.0:1789760713.482550:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17342464 -> 17346560 00010000:00010000:3.0:1789760713.482623:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482637:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482646:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482653:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.482657:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482665:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482685:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.482708:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7001 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17342464->17346559] (req 17342464->17346559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.482847:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507169536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44493us (44862us total) trans 4294971554 rc 0/0 00000100:00100000:3.0:1789760713.482861:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4264 00000100:00100000:3.0:1789760713.482871:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694702073098 time 694753640793 state 0x2 ref 0 00000100:00100000:2.0:1789760713.487924:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.488119:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169664 00000100:00100000:1.0:1789760713.488129:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169664 00000100:00100000:1.0:1789760713.488187:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694702073098 time 694753640793 state 0x0 ref 1 00000100:00100000:1.0:1789760713.488203:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694753901209 time 694753640793 state 0x1 ref 1 00000100:00100000:1.0:1789760713.488268:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694753901209 time 694803570664 state 0x0 ref 1 00000100:00100000:1.0:1789760713.488280:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507169664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.488331:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17346560, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.488340:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.488354:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760713.488359:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.488365:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.488389:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.521746:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971555 is committed 00002000:00010000:2.0:1789760713.522279:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17346560 -> 17350656 00010000:00010000:2.0:1789760713.522296:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522311:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522320:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522326:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760713.522331:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522342:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522363:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.522369:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7008 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17346560->17350655] (req 17346560->17350655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.522483:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507169664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34200us (34565us total) trans 4294971555 rc 0/0 00000100:00100000:2.0:1789760713.522494:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4265 00000100:00100000:2.0:1789760713.522502:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694753901209 time 694803570664 state 0x2 ref 0 00000100:00100000:3.0:1789760713.527644:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.528371:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169792 00000100:00100000:2.0:1789760713.528389:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169792 00000100:00100000:2.0:1789760713.528456:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694753901209 time 694803570664 state 0x0 ref 1 00000100:00100000:2.0:1789760713.528474:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694803831080 time 694803570664 state 0x1 ref 1 00000100:00100000:2.0:1789760713.528510:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694803831080 time 694843813053 state 0x0 ref 1 00000100:00100000:2.0:1789760713.528523:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507169792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.528569:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17350656, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.528578:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.528595:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.528599:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.528606:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.528620:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.583783:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971556 is committed 00002000:00010000:3.0:1789760713.584104:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17350656 -> 17354752 00010000:00010000:3.0:1789760713.584123:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584137:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584146:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584153:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.584159:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584168:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584192:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.584200:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b700f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17350656->17354751] (req 17350656->17354751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.584429:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507169792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55901us (56788us total) trans 4294971556 rc 0/0 00000100:00100000:3.0:1789760713.584440:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4266 00000100:00100000:3.0:1789760713.584452:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694803831080 time 694843813053 state 0x2 ref 0 00000100:00100000:3.0:1789760713.587620:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.587737:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507169920 00000100:00100000:2.0:1789760713.587746:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507169920 00000100:00100000:2.0:1789760713.587802:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694803831080 time 694843813053 state 0x0 ref 1 00000100:00100000:2.0:1789760713.587818:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694844073469 time 694843813053 state 0x1 ref 1 00000100:00100000:2.0:1789760713.587843:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694844073469 time 694903147149 state 0x0 ref 1 00000100:00100000:2.0:1789760713.587855:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.587923:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17354752, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.587933:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.587949:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.587954:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.587960:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.587977:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.615933:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971557 is committed 00002000:00010000:3.0:1789760713.616065:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17354752 -> 17358848 00010000:00010000:3.0:1789760713.616080:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616092:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616101:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616109:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.616114:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616124:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616143:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.616155:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7016 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17354752->17358847] (req 17354752->17358847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.616248:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507169920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28389us (28631us total) trans 4294971557 rc 0/0 00000100:00100000:3.0:1789760713.616258:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4267 00000100:00100000:3.0:1789760713.616277:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694844073469 time 694903147149 state 0x2 ref 0 00000100:00100000:2.0:1789760713.622147:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.622238:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170048 00000100:00100000:1.0:1789760713.622248:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170048 00000100:00100000:1.0:1789760713.622320:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694844073469 time 694903147149 state 0x0 ref 1 00000100:00100000:1.0:1789760713.622336:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694903407565 time 694903147149 state 0x1 ref 1 00000100:00100000:1.0:1789760713.622366:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694903407565 time 694937669920 state 0x0 ref 1 00000100:00100000:1.0:1789760713.622389:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.622439:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17358848, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.622448:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.622463:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760713.622468:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.622474:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.622487:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.629984:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760713.659161:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971558 is committed 00002000:00010000:2.0:1789760713.659244:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17358848 -> 17362944 00010000:00010000:2.0:1789760713.659258:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659339:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659349:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659356:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760713.659361:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659370:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659521:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.659530:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b701d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17358848->17362943] (req 17358848->17362943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.659660:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37269us (37518us total) trans 4294971558 rc 0/0 00000100:00100000:2.0:1789760713.659671:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4268 00000100:00100000:2.0:1789760713.659680:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694903407565 time 694937669920 state 0x2 ref 0 00000100:00100000:3.0:1789760713.666609:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760713.666763:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170176 00000100:00100000:2.0:1789760713.666773:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170176 00000100:00100000:2.0:1789760713.666828:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694903407565 time 694937669920 state 0x0 ref 1 00000100:00100000:2.0:1789760713.666843:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694937930336 time 694937669920 state 0x1 ref 1 00000100:00100000:2.0:1789760713.666915:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694937930336 time 694982217656 state 0x0 ref 1 00000100:00100000:2.0:1789760713.666928:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760713.666990:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17362944, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:2.0:1789760713.667008:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.667023:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760713.667028:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.667042:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.667058:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.711173:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971559 is committed 00002000:00010000:3.0:1789760713.711286:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17362944 -> 17367040 00010000:00010000:3.0:1789760713.711301:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711314:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711323:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711329:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760713.711333:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711342:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711363:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.711370:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7024 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17362944->17367039] (req 17362944->17367039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.711632:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44701us (45027us total) trans 4294971559 rc 0/0 00000100:00100000:3.0:1789760713.711643:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4269 00000100:00100000:3.0:1789760713.711653:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694937930336 time 694982217656 state 0x2 ref 0 00000100:00100000:1.0:1789760713.715564:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760713.715654:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170304 00000100:00100000:3.0:1789760713.715666:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170304 00000100:00100000:3.0:1789760713.715819:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694937930336 time 694982217656 state 0x0 ref 1 00000100:00100000:3.0:1789760713.715834:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694982478072 time 694982217656 state 0x1 ref 1 00000100:00100000:3.0:1789760713.715890:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694982478072 time 695031193335 state 0x0 ref 1 00000100:00100000:3.0:1789760713.715903:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507170304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760713.715954:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17367040, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:3.0:1789760713.715963:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.715977:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760713.715982:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.715989:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.716012:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.760615:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971560 is committed 00002000:00010000:1.0:1789760713.760738:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17367040 -> 17371136 00010000:00010000:1.0:1789760713.760754:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760767:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760775:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760781:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.760785:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760793:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760813:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.760820:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b702b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17367040->17371135] (req 17367040->17371135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.760917:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507170304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45011us (45357us total) trans 4294971560 rc 0/0 00000100:00100000:1.0:1789760713.760927:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4270 00000100:00100000:1.0:1789760713.760937:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694982478072 time 695031193335 state 0x2 ref 0 00000100:00100000:3.0:1789760713.764610:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.764823:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170432 00000100:00100000:1.0:1789760713.764834:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170432 00000100:00100000:1.0:1789760713.764888:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 694982478072 time 695031193335 state 0x0 ref 1 00000100:00100000:1.0:1789760713.764902:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695031453751 time 695031193335 state 0x1 ref 1 00000100:00100000:1.0:1789760713.764922:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695031453751 time 695080226112 state 0x0 ref 1 00000100:00100000:1.0:1789760713.764934:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.764980:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17371136, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.764990:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.765014:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760713.765020:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.765027:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.765050:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.801348:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971561 is committed 00002000:00010000:1.0:1789760713.801377:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17371136 -> 17375232 00010000:00010000:1.0:1789760713.801392:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801404:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801418:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801427:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.801431:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801441:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801460:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.801468:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7032 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17371136->17375231] (req 17371136->17375231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.801561:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36623us (36954us total) trans 4294971561 rc 0/0 00000100:00100000:1.0:1789760713.801571:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4271 00000100:00100000:1.0:1789760713.801580:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695031453751 time 695080226112 state 0x2 ref 0 00000100:00100000:2.0:1789760713.805365:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760713.805514:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170560 00000100:00100000:3.0:1789760713.805524:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170560 00000100:00100000:3.0:1789760713.805578:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695031453751 time 695080226112 state 0x0 ref 1 00000100:00100000:3.0:1789760713.805593:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695080486528 time 695080226112 state 0x1 ref 1 00000100:00100000:3.0:1789760713.805628:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695080486528 time 695120930640 state 0x0 ref 1 00000100:00100000:3.0:1789760713.805640:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507170560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760713.805686:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17375232, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:3.0:1789760713.805718:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.805736:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760713.805741:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.805747:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.805761:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.826026:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971562 is committed 00002000:00010000:1.0:1789760713.826191:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17375232 -> 17379328 00010000:00010000:1.0:1789760713.826207:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826226:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826234:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826240:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.826245:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826253:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826382:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.826391:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7039 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17375232->17379327] (req 17375232->17379327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.826485:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507170560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 20841us (21126us total) trans 4294971562 rc 0/0 00000100:00100000:1.0:1789760713.826495:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4272 00000100:00100000:1.0:1789760713.826503:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695080486528 time 695120930640 state 0x2 ref 0 00000100:00100000:0.0:1789760713.830209:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.830465:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170688 00000100:00100000:1.0:1789760713.830478:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170688 00000100:00100000:1.0:1789760713.830526:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695080486528 time 695120930640 state 0x0 ref 1 00000100:00100000:1.0:1789760713.830539:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695121191056 time 695120930640 state 0x1 ref 1 00000100:00100000:1.0:1789760713.830560:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695121191056 time 695145863848 state 0x0 ref 1 00000100:00100000:1.0:1789760713.830569:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507170688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.830615:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17379328, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.830623:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.830638:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760713.830642:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.830647:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.830659:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760713.851614:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971563 is committed 00002000:00010000:1.0:1789760713.851750:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17379328 -> 17383424 00010000:00010000:1.0:1789760713.851765:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851776:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851791:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851797:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.851802:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851810:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851830:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.851836:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7040 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17379328->17383423] (req 17379328->17383423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.851925:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507170688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21351us (21720us total) trans 4294971563 rc 0/0 00000100:00100000:1.0:1789760713.851936:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4273 00000100:00100000:1.0:1789760713.851946:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695121191056 time 695145863848 state 0x2 ref 0 00000100:00100000:3.0:1789760713.857451:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.857615:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170816 00000100:00100000:1.0:1789760713.857626:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170816 00000100:00100000:1.0:1789760713.857678:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695121191056 time 695145863848 state 0x0 ref 1 00000100:00100000:1.0:1789760713.857727:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695146124264 time 695145863848 state 0x1 ref 1 00000100:00100000:1.0:1789760713.857747:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695146124264 time 695173051394 state 0x0 ref 1 00000100:00100000:1.0:1789760713.857760:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.857809:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17383424, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.857818:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857834:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760713.857840:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.857847:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857862:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760713.906788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971564 is committed 00002000:00010000:1.0:1789760713.906911:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17383424 -> 17387520 00010000:00010000:1.0:1789760713.906926:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.906939:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.906950:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.906956:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.906962:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.906972:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.906992:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.907017:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7047 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.907135:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507170816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49372us (49688us total) trans 4294971564 rc 0/0 00000100:00100000:1.0:1789760713.907146:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4274 00000100:00100000:1.0:1789760713.907155:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695146124264 time 695173051394 state 0x2 ref 0 00000100:00100000:2.0:1789760713.912456:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.912603:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507170944 00000100:00100000:1.0:1789760713.912613:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507170944 00000100:00100000:1.0:1789760713.912667:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695146124264 time 695173051394 state 0x0 ref 1 00000100:00100000:1.0:1789760713.912682:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695173311810 time 695173051394 state 0x1 ref 1 00000100:00100000:1.0:1789760713.912748:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695173311810 time 695228051053 state 0x0 ref 1 00000100:00100000:1.0:1789760713.912759:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507170944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.912809:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17387520, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.912819:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.912834:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760713.912838:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.912844:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.912858:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.954295:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104734a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538261504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760713.954370:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760713.955304:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261504 00000100:00100000:0.0:1789760713.955315:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261504 00000100:00100000:0.0:1789760713.955371:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:0.0:1789760713.955382:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538261504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760713.955408:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3840933888 avail 3815768064 00000100:00100000:0.0:1789760713.955472:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538261504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (1102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760713.955479:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:0.0:1789760713.955789:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104734a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538261504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760713.960449:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538261632:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760713.960524:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760713.960842:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261632 00000100:00100000:0.0:1789760713.960851:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261632 00000100:00100000:0.0:1789760713.960897:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:0.0:1789760713.960906:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538261632:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760713.960928:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3840933888 avail 3815768064 00000100:00100000:0.0:1789760713.963580:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538261632:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2666us (3047us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760713.963593:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1789760713.963894:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538261632:0@lo:13:osp-pre-0-1.0 00000001:00080000:2.0:1789760713.974789:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971565 is committed 00002000:00010000:1.0:1789760713.974870:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17387520 -> 17391616 00010000:00010000:1.0:1789760713.974896:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974911:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974921:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974929:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760713.974934:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974944:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974970:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.974978:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b704e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17387520->17391615] (req 17387520->17391615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.975082:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507170944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62319us (62631us total) trans 4294971565 rc 0/0 00000100:00100000:1.0:1789760713.975092:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4275 00000100:00100000:1.0:1789760713.975102:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695173311810 time 695228051053 state 0x2 ref 0 00000100:00100000:3.0:1789760713.983301:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760713.983472:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171072 00000100:00100000:1.0:1789760713.983482:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171072 00000100:00100000:1.0:1789760713.983545:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695173311810 time 695228051053 state 0x0 ref 1 00000100:00100000:1.0:1789760713.983568:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695228311469 time 695228051053 state 0x1 ref 1 00000100:00100000:1.0:1789760713.983605:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695228311469 time 695298905255 state 0x0 ref 1 00000100:00100000:1.0:1789760713.983617:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760713.983665:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17391616, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760713.983679:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.983741:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760713.983747:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.983756:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.983774:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.019651:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971566 is committed 00002000:00010000:2.0:1789760714.021158:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17391616 -> 17395712 00010000:00010000:2.0:1789760714.021179:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021193:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021202:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021209:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.021214:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021222:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021246:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.021254:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7055 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.021441:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37820us (38144us total) trans 4294971566 rc 0/0 00000100:00100000:2.0:1789760714.021451:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4276 00000100:00100000:2.0:1789760714.021460:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695228311469 time 695298905255 state 0x2 ref 0 00000100:00100000:3.0:1789760714.033800:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.035234:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171200 00000100:00100000:1.0:1789760714.035248:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171200 00000100:00100000:1.0:1789760714.035309:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695228311469 time 695298905255 state 0x0 ref 1 00000100:00100000:1.0:1789760714.035327:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695299165671 time 695298905255 state 0x1 ref 1 00000100:00100000:1.0:1789760714.035363:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695299165671 time 695350665800 state 0x0 ref 1 00000100:00100000:1.0:1789760714.035375:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.035426:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17395712, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x8807 00010000:00010000:1.0:1789760714.035435:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.035450:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.035455:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.035461:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.035474:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.088368:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971567 is committed 00002000:00010000:2.0:1789760714.088497:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17395712 -> 17399808 00002000:00010000:2.0:1789760714.088502:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760712 -> 1789760713 00002000:00010000:2.0:1789760714.088505:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760712 -> 1789760713 00010000:00010000:2.0:1789760714.088586:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088599:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088608:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088615:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.088619:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088629:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088658:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.088665:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b705c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17395712->17399807] (req 17395712->17399807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.088785:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53406us (54989us total) trans 4294971567 rc 0/0 00000100:00100000:2.0:1789760714.088795:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4277 00000100:00100000:2.0:1789760714.088805:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695299165671 time 695350665800 state 0x2 ref 0 00000100:00100000:3.0:1789760714.097517:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.097616:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171328 00000100:00100000:1.0:1789760714.097627:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171328 00000100:00100000:1.0:1789760714.097679:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695299165671 time 695350665800 state 0x0 ref 1 00000100:00100000:1.0:1789760714.097752:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695350926216 time 695350665800 state 0x1 ref 1 00000100:00100000:1.0:1789760714.097774:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695350926216 time 695413078004 state 0x0 ref 1 00000100:00100000:1.0:1789760714.097788:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.097838:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17399808, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.097847:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.097863:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.097867:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.097873:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.097892:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.130209:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17399808 -> 17403904 00010000:00010000:3.0:1789760714.130229:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130242:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130250:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130257:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760714.130261:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130270:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130291:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.130298:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7063 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17399808->17403903] (req 17399808->17403903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.130394:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507171328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32600us (32926us total) trans 4294971568 rc 0/0 00000100:00100000:3.0:1789760714.130405:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4278 00000100:00100000:3.0:1789760714.130414:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695350926216 time 695413078004 state 0x2 ref 0 00000001:00080000:0.0:1789760714.130420:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971568 is committed 00000100:00100000:3.0:1789760714.133674:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760714.133779:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171456 00000100:00100000:2.0:1789760714.133788:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171456 00000100:00100000:2.0:1789760714.133885:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695350926216 time 695413078004 state 0x0 ref 1 00000100:00100000:2.0:1789760714.133896:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695413338420 time 695413078004 state 0x1 ref 1 00000100:00100000:2.0:1789760714.133918:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695413338420 time 695449221948 state 0x0 ref 1 00000100:00100000:2.0:1789760714.133931:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760714.133974:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17403904, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:2.0:1789760714.133982:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.134007:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760714.134012:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.134019:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.134035:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.166244:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971569 is committed 00002000:00010000:2.0:1789760714.166396:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17403904 -> 17408000 00010000:00010000:2.0:1789760714.166413:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166426:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166434:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166441:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.166446:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166455:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166475:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.166482:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b706a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17403904->17407999] (req 17403904->17407999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.166601:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32667us (32931us total) trans 4294971569 rc 0/0 00000100:00100000:2.0:1789760714.166612:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4279 00000100:00100000:2.0:1789760714.166621:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695413338420 time 695449221948 state 0x2 ref 0 00000100:00100000:1.0:1789760714.170007:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760714.170142:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171584 00000100:00100000:2.0:1789760714.170152:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171584 00000100:00100000:2.0:1789760714.170211:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695413338420 time 695449221948 state 0x0 ref 1 00000100:00100000:2.0:1789760714.170228:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695449482364 time 695449221948 state 0x1 ref 1 00000100:00100000:2.0:1789760714.170260:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695449482364 time 695485563706 state 0x0 ref 1 00000100:00100000:2.0:1789760714.170281:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760714.170331:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17408000, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:2.0:1789760714.170340:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.170355:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760714.170360:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.170366:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.170385:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.228232:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971570 is committed 00002000:00010000:1.0:1789760714.228538:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17408000 -> 17412096 00010000:00010000:1.0:1789760714.228553:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228566:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228574:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228734:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760714.228740:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228751:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228771:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.228779:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7071 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17408000->17412095] (req 17408000->17412095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.229313:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58632us (58926us total) trans 4294971570 rc 0/0 00000100:00100000:1.0:1789760714.229325:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4280 00000100:00100000:1.0:1789760714.229335:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695449482364 time 695485563706 state 0x2 ref 0 00000100:00100000:3.0:1789760714.236111:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.236182:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171712 00000100:00100000:1.0:1789760714.236191:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171712 00000100:00100000:1.0:1789760714.236255:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695449482364 time 695485563706 state 0x0 ref 1 00000100:00100000:1.0:1789760714.236269:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695485824122 time 695485563706 state 0x1 ref 1 00000100:00100000:1.0:1789760714.236301:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695485824122 time 695551604816 state 0x0 ref 1 00000100:00100000:1.0:1789760714.236313:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507171712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.236361:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17412096, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.236370:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.236385:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.236389:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.236395:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.236409:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.276055:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17412096 -> 17416192 00010000:00010000:1.0:1789760714.276075:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276088:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276101:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276107:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760714.276112:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276121:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276142:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.276150:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7078 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17412096->17416191] (req 17412096->17416191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.276742:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507171712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40425us (40635us total) trans 4294971571 rc 0/0 00000100:00100000:1.0:1789760714.276753:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4281 00000100:00100000:1.0:1789760714.276763:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695485824122 time 695551604816 state 0x2 ref 0 00000001:00080000:3.0:1789760714.276856:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971571 is committed 00000100:00100000:2.0:1789760714.281041:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.281167:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171840 00000100:00100000:1.0:1789760714.281176:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171840 00000100:00100000:1.0:1789760714.281238:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695485824122 time 695551604816 state 0x0 ref 1 00000100:00100000:1.0:1789760714.281255:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695551865232 time 695551604816 state 0x1 ref 1 00000100:00100000:1.0:1789760714.281284:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695551865232 time 695596587190 state 0x0 ref 1 00000100:00100000:1.0:1789760714.281296:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.281341:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17416192, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.281351:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.281367:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.281372:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.281379:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.281392:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.344794:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971572 is committed 00002000:00010000:2.0:1789760714.345196:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17416192 -> 17420288 00010000:00010000:2.0:1789760714.345229:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345242:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345251:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345259:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.345264:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345274:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345295:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.345302:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b707f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17416192->17420287] (req 17416192->17420287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.345396:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64098us (64368us total) trans 4294971572 rc 0/0 00000100:00100000:2.0:1789760714.345406:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4282 00000100:00100000:2.0:1789760714.345415:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695551865232 time 695596587190 state 0x2 ref 0 00000100:00100000:2.0:1789760714.354138:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.354457:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507171968 00000100:00100000:1.0:1789760714.354469:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507171968 00000100:00100000:1.0:1789760714.354530:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695551865232 time 695596587190 state 0x0 ref 1 00000100:00100000:1.0:1789760714.354549:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695596847606 time 695596587190 state 0x1 ref 1 00000100:00100000:1.0:1789760714.354581:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695596847606 time 695669884035 state 0x0 ref 1 00000100:00100000:1.0:1789760714.354594:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507171968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.354647:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17420288, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.354658:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354675:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.354680:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.354686:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354738:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.390487:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17420288 -> 17424384 00010000:00010000:0.0:1789760714.390505:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390517:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390525:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390531:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.390536:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390543:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390561:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.390568:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7086 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17420288->17424383] (req 17420288->17424383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.390651:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507171968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36055us (36519us total) trans 4294971573 rc 0/0 00000100:00100000:0.0:1789760714.390660:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4283 00000100:00100000:0.0:1789760714.390669:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695596847606 time 695669884035 state 0x2 ref 0 00000001:00080000:0.0:1789760714.390926:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971573 is committed 00000100:00100000:3.0:1789760714.407680:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.407948:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172096 00000100:00100000:1.0:1789760714.407959:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172096 00000100:00100000:1.0:1789760714.408519:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695596847606 time 695669884035 state 0x0 ref 1 00000100:00100000:1.0:1789760714.408536:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695670144451 time 695669884035 state 0x1 ref 1 00000100:00100000:1.0:1789760714.408571:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695670144451 time 695723873850 state 0x0 ref 1 00000100:00100000:1.0:1789760714.408586:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507172096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.408638:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17424384, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.408647:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.408662:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.408666:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.408673:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.408689:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.455455:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971574 is committed 00002000:00010000:0.0:1789760714.455825:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17424384 -> 17428480 00010000:00010000:0.0:1789760714.455838:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455847:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455854:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455858:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.455862:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455869:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455883:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.455887:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b708d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17424384->17428479] (req 17424384->17428479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.456034:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507172096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47439us (48353us total) trans 4294971574 rc 0/0 00000100:00100000:0.0:1789760714.456046:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4284 00000100:00100000:0.0:1789760714.456056:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695670144451 time 695723873850 state 0x2 ref 0 00000100:00100000:3.0:1789760714.460082:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538261760:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760714.460170:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760714.460345:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261760 00000100:00100000:2.0:1789760714.460353:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261760 00000100:00100000:2.0:1789760714.460689:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760714.460730:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538261760:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760714.460776:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760714.460828:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538261760:12345-0@lo:41:osp-pre-1-0.0 Request processed in 100us (657us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760714.460835:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:0.0:1789760714.460931:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538261760:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760714.479269:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.480383:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172224 00000100:00100000:1.0:1789760714.480396:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172224 00000100:00100000:1.0:1789760714.480469:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695670144451 time 695723873850 state 0x0 ref 1 00000100:00100000:1.0:1789760714.480548:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695724134266 time 695723873850 state 0x1 ref 1 00000100:00100000:1.0:1789760714.480636:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695724134266 time 695795937948 state 0x0 ref 1 00000100:00100000:1.0:1789760714.480651:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.480745:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17428480, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.480754:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.480770:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.480775:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.480782:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.480796:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.527211:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971575 is committed 00002000:00010000:0.0:1789760714.527468:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17428480 -> 17432576 00010000:00010000:0.0:1789760714.527483:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527496:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527506:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527513:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.527518:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527527:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527547:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.527560:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7094 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17428480->17432575] (req 17428480->17432575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.527640:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46989us (48377us total) trans 4294971575 rc 0/0 00000100:00100000:0.0:1789760714.527649:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4285 00000100:00100000:0.0:1789760714.527658:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695724134266 time 695795937948 state 0x2 ref 0 00000100:00100000:2.0:1789760714.531730:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.531976:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172352 00000100:00100000:0.0:1789760714.531986:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172352 00000100:00100000:0.0:1789760714.532051:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695724134266 time 695795937948 state 0x0 ref 1 00000100:00100000:0.0:1789760714.532066:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695796198364 time 695795937948 state 0x1 ref 1 00000100:00100000:1.0:1789760714.532177:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695796198364 time 695847478416 state 0x0 ref 1 00000100:00100000:1.0:1789760714.532191:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.532235:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17432576, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.532243:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.532259:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760714.532264:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.532270:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.532284:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.551376:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971576 is committed 00002000:00010000:0.0:1789760714.551511:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17432576 -> 17436672 00010000:00010000:0.0:1789760714.551527:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551541:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551551:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551559:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.551564:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551573:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551590:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.551594:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b709b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17432576->17436671] (req 17432576->17436671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.551673:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 19478us (19974us total) trans 4294971576 rc 0/0 00000100:00100000:0.0:1789760714.551684:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4286 00000100:00100000:0.0:1789760714.551731:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695796198364 time 695847478416 state 0x2 ref 0 00000100:00100000:3.0:1789760714.555130:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.555226:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172480 00000100:00100000:0.0:1789760714.555236:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172480 00000100:00100000:0.0:1789760714.555466:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695796198364 time 695847478416 state 0x0 ref 1 00000100:00100000:0.0:1789760714.555485:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695847738832 time 695847478416 state 0x1 ref 1 00000100:00100000:0.0:1789760714.555529:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695847738832 time 695870831922 state 0x0 ref 1 00000100:00100000:0.0:1789760714.555543:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507172480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.555594:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17436672, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.555602:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.555619:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760714.555624:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.555630:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.555650:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.583294:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17436672 -> 17440768 00010000:00010000:3.0:1789760714.583315:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583328:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583343:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583350:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760714.583355:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583365:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583386:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.583394:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b70a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17436672->17440767] (req 17436672->17440767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.583464:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971577 is committed 00000100:00100000:3.0:1789760714.583496:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507172480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27952us (28371us total) trans 4294971577 rc 0/0 00000100:00100000:3.0:1789760714.584608:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4287 00000100:00100000:3.0:1789760714.584620:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695847738832 time 695870831922 state 0x2 ref 0 00000100:00100000:2.0:1789760714.586451:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.586673:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172608 00000100:00100000:0.0:1789760714.586683:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172608 00000100:00100000:0.0:1789760714.586770:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695847738832 time 695870831922 state 0x0 ref 1 00000100:00100000:0.0:1789760714.586785:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695871092338 time 695870831922 state 0x1 ref 1 00000100:00100000:0.0:1789760714.586806:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695871092338 time 695902109815 state 0x0 ref 1 00000100:00100000:0.0:1789760714.586818:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507172608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.586862:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17440768, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.586871:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.586886:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760714.586890:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.586897:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.586917:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.608730:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971578 is committed 00002000:00010000:2.0:1789760714.608788:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17440768 -> 17444864 00010000:00010000:2.0:1789760714.608805:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.608818:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.608834:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.608841:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.608845:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.608853:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.609356:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.609367:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b70a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17440768->17444863] (req 17440768->17444863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.609767:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507172608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22945us (23317us total) trans 4294971578 rc 0/0 00000100:00100000:2.0:1789760714.609778:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4288 00000100:00100000:2.0:1789760714.609788:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695871092338 time 695902109815 state 0x2 ref 0 00000100:00100000:1.0:1789760714.615284:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.615391:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172736 00000100:00100000:0.0:1789760714.615400:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172736 00000100:00100000:0.0:1789760714.615514:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695871092338 time 695902109815 state 0x0 ref 1 00000100:00100000:0.0:1789760714.615530:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695902370231 time 695902109815 state 0x1 ref 1 00000100:00100000:0.0:1789760714.615601:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695902370231 time 695930902956 state 0x0 ref 1 00000100:00100000:0.0:1789760714.615615:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.615664:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17444864, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.615672:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.615707:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760714.615713:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.615720:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.615734:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.646237:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971579 is committed 00002000:00010000:0.0:1789760714.646319:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17444864 -> 17448960 00010000:00010000:0.0:1789760714.646334:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646347:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646356:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646364:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.646368:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646377:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646393:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.646400:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.646552:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30933us (31270us total) trans 4294971579 rc 0/0 00000100:00100000:0.0:1789760714.646563:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4289 00000100:00100000:0.0:1789760714.646571:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695902370231 time 695930902956 state 0x2 ref 0 00010000:00010000:2.0:1789760714.651891:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760714.654254:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.654555:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172864 00000100:00100000:0.0:1789760714.654565:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172864 00000100:00100000:0.0:1789760714.654620:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695902370231 time 695930902956 state 0x0 ref 1 00000100:00100000:0.0:1789760714.654636:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695931163372 time 695930902956 state 0x1 ref 1 00000100:00100000:0.0:1789760714.654670:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695931163372 time 695969973152 state 0x0 ref 1 00000100:00100000:0.0:1789760714.654682:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507172864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.654858:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17448960, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.654869:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.654885:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760714.654890:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.654896:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.654911:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.720267:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971580 is committed 00002000:00010000:0.0:1789760714.720379:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17448960 -> 17453056 00010000:00010000:0.0:1789760714.720394:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720407:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720417:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720425:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.720429:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720436:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720451:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.720456:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17448960->17453055] (req 17448960->17453055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.720537:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507172864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65855us (66289us total) trans 4294971580 rc 0/0 00000100:00100000:0.0:1789760714.720544:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4290 00000100:00100000:0.0:1789760714.720552:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695931163372 time 695969973152 state 0x2 ref 0 00000100:00100000:1.0:1789760714.723871:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760714.723971:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507172992 00000100:00100000:2.0:1789760714.723981:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507172992 00000100:00100000:2.0:1789760714.724047:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695931163372 time 695969973152 state 0x0 ref 1 00000100:00100000:2.0:1789760714.724066:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695970233568 time 695969973152 state 0x1 ref 1 00000100:00100000:2.0:1789760714.724097:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695970233568 time 696039400049 state 0x0 ref 1 00000100:00100000:2.0:1789760714.724109:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760714.725421:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17453056, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:2.0:1789760714.725436:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.725454:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760714.725459:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.725466:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.725481:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.752499:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17453056 -> 17457152 00010000:00010000:0.0:1789760714.752521:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752538:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752546:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752553:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.752557:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752566:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752584:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.752591:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b70be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.752633:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971581 is committed 00000100:00100000:0.0:1789760714.752681:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507172992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28570us (28816us total) trans 4294971581 rc 0/0 00000100:00100000:0.0:1789760714.752689:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4291 00000100:00100000:0.0:1789760714.752738:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695970233568 time 696039400049 state 0x2 ref 0 00000100:00100000:1.0:1789760714.760080:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.760181:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173120 00000100:00100000:0.0:1789760714.760192:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173120 00000100:00100000:0.0:1789760714.760255:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 695970233568 time 696039400049 state 0x0 ref 1 00000100:00100000:0.0:1789760714.760270:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696039660465 time 696039400049 state 0x1 ref 1 00000100:00100000:0.0:1789760714.760772:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696039660465 time 696076073569 state 0x0 ref 1 00000100:00100000:0.0:1789760714.760784:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507173120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.760837:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17457152, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.760846:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.760863:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760714.760867:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.760874:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.760888:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.803363:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971582 is committed 00002000:00010000:0.0:1789760714.803592:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17457152 -> 17461248 00010000:00010000:0.0:1789760714.803848:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803862:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803876:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803884:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.803889:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803898:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803918:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.803927:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.804164:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507173120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43376us (44089us total) trans 4294971582 rc 0/0 00000100:00100000:0.0:1789760714.804174:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4292 00000100:00100000:0.0:1789760714.804184:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696039660465 time 696076073569 state 0x2 ref 0 00000100:00100000:3.0:1789760714.815833:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.816938:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173248 00000100:00100000:0.0:1789760714.816948:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173248 00000100:00100000:0.0:1789760714.817012:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696039660465 time 696076073569 state 0x0 ref 1 00000100:00100000:0.0:1789760714.817028:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696076333985 time 696076073569 state 0x1 ref 1 00000100:00100000:0.0:1789760714.817062:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696076333985 time 696132364983 state 0x0 ref 1 00000100:00100000:0.0:1789760714.817075:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507173248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760714.817120:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17461248, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:0.0:1789760714.817127:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.817140:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760714.817143:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.817148:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.817158:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.853039:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971583 is committed 00002000:00010000:3.0:1789760714.853164:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17461248 -> 17465344 00010000:00010000:3.0:1789760714.853178:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853315:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853324:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853331:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760714.853405:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853416:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853435:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.853443:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b70cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17461248->17465343] (req 17461248->17465343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.853678:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507173248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36600us (37851us total) trans 4294971583 rc 0/0 00000100:00100000:3.0:1789760714.853689:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4293 00000100:00100000:3.0:1789760714.853805:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696076333985 time 696132364983 state 0x2 ref 0 00000100:00100000:0.0:1789760714.857975:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.858164:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173376 00000100:00100000:1.0:1789760714.858174:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173376 00000100:00100000:1.0:1789760714.858226:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696076333985 time 696132364983 state 0x0 ref 1 00000100:00100000:1.0:1789760714.858241:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696132625399 time 696132364983 state 0x1 ref 1 00000100:00100000:1.0:1789760714.858306:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696132625399 time 696173607678 state 0x0 ref 1 00000100:00100000:1.0:1789760714.858323:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.858380:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17465344, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.858388:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.858401:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760714.858405:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.858411:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.858424:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.894414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971584 is committed 00002000:00010000:1.0:1789760714.894576:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17465344 -> 17469440 00010000:00010000:1.0:1789760714.894600:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894613:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894623:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894630:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760714.894634:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894643:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894662:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.894670:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b70d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.894837:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36511us (36865us total) trans 4294971584 rc 0/0 00000100:00100000:1.0:1789760714.894848:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4294 00000100:00100000:1.0:1789760714.894858:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696132625399 time 696173607678 state 0x2 ref 0 00000100:00100000:1.0:1789760714.906569:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760714.906814:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173504 00000100:00100000:0.0:1789760714.906824:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173504 00000100:00100000:2.0:1789760714.906967:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696132625399 time 696173607678 state 0x0 ref 1 00000100:00100000:2.0:1789760714.906981:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696173868094 time 696173607678 state 0x1 ref 1 00000100:00100000:2.0:1789760714.907070:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696173868094 time 696222372410 state 0x0 ref 1 00000100:00100000:2.0:1789760714.907083:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760714.907129:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17469440, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:2.0:1789760714.907137:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.907152:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760714.907156:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.907163:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.907177:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.933306:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971585 is committed 00002000:00010000:2.0:1789760714.933379:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17469440 -> 17473536 00010000:00010000:2.0:1789760714.933395:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933406:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933413:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933417:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760714.933420:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933427:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933443:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.933448:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.933561:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26475us (26996us total) trans 4294971585 rc 0/0 00000100:00100000:2.0:1789760714.933570:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4295 00000100:00100000:2.0:1789760714.933581:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696173868094 time 696222372410 state 0x2 ref 0 00000100:00100000:0.0:1789760714.936558:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760714.937218:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173632 00000100:00100000:3.0:1789760714.937226:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173632 00000100:00100000:3.0:1789760714.937274:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696173868094 time 696222372410 state 0x0 ref 1 00000100:00100000:3.0:1789760714.937286:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696222632826 time 696222372410 state 0x1 ref 1 00000100:00100000:3.0:1789760714.937336:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696222632826 time 696252640319 state 0x0 ref 1 00000100:00100000:3.0:1789760714.937345:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760714.937386:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17473536, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:3.0:1789760714.937393:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.937407:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760714.937411:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.937415:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.937425:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.955234:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971586 is committed 00002000:00010000:0.0:1789760714.955299:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17473536 -> 17477632 00010000:00010000:0.0:1789760714.955313:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955325:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955338:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955345:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760714.955359:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955368:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955387:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.955395:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b70e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17473536->17477631] (req 17473536->17477631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.955505:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 18156us (18953us total) trans 4294971586 rc 0/0 00000100:00100000:0.0:1789760714.955512:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4296 00000100:00100000:0.0:1789760714.955520:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696222632826 time 696252640319 state 0x2 ref 0 00000100:00100000:0.0:1789760714.958146:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760714.958239:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173760 00000100:00100000:1.0:1789760714.958248:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173760 00000100:00100000:1.0:1789760714.958298:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696222632826 time 696252640319 state 0x0 ref 1 00000100:00100000:1.0:1789760714.958310:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696252900735 time 696252640319 state 0x1 ref 1 00000100:00100000:1.0:1789760714.958329:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696252900735 time 696273633404 state 0x0 ref 1 00000100:00100000:1.0:1789760714.958336:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760714.958392:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17477632, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:1.0:1789760714.958401:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.958414:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760714.958418:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.958422:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.958434:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.971876:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538261888:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760714.971936:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760714.971969:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538261888 00000100:00100000:1.0:1789760714.971977:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538261888 00000100:00100000:1.0:1789760714.972205:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1789760714.972216:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538261888:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760714.972251:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760714.972392:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538261888:12345-0@lo:41:osp-pre-0-1.0 Request processed in 177us (454us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760714.972400:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:0.0:1789760714.972406:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538261888:0@lo:41:osp-pre-0-1.0 00002000:00010000:1.0:1789760714.982064:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17477632 -> 17481728 00010000:00010000:1.0:1789760714.982083:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982097:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982113:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982121:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760714.982126:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982135:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982154:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.982163:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b70e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.982176:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971587 is committed 00000100:00100000:1.0:1789760714.982284:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23941us (24137us total) trans 4294971587 rc 0/0 00000100:00100000:1.0:1789760714.982295:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4297 00000100:00100000:1.0:1789760714.982304:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696252900735 time 696273633404 state 0x2 ref 0 00000100:00100000:1.0:1789760714.990647:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760714.990823:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507173888 00000100:00100000:2.0:1789760714.990834:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507173888 00000100:00100000:2.0:1789760714.990889:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696252900735 time 696273633404 state 0x0 ref 1 00000100:00100000:2.0:1789760714.990905:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696273893820 time 696273633404 state 0x1 ref 1 00000100:00100000:2.0:1789760714.990939:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696273893820 time 696306242448 state 0x0 ref 1 00000100:00100000:2.0:1789760714.990965:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507173888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760714.991541:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17481728, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x8807 00010000:00010000:2.0:1789760714.991552:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.991573:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760714.991578:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.991585:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.991600:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760715.033462:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17481728 -> 17485824 00002000:00010000:3.0:1789760715.033472:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:3.0:1789760715.033475:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:3.0:1789760715.033488:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033564:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033573:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033580:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760715.033585:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033594:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033613:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.033620:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b70ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.033736:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507173888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42782us (43095us total) trans 4294971588 rc 0/0 00000100:00100000:3.0:1789760715.033746:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4298 00000100:00100000:3.0:1789760715.033756:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696273893820 time 696306242448 state 0x2 ref 0 00000001:00080000:1.0:1789760715.034035:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971588 is committed 00000100:00100000:0.0:1789760715.037247:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760715.037329:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174016 00000100:00100000:3.0:1789760715.037339:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174016 00000100:00100000:3.0:1789760715.037406:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696273893820 time 696306242448 state 0x0 ref 1 00000100:00100000:3.0:1789760715.037421:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696306502864 time 696306242448 state 0x1 ref 1 00000100:00100000:3.0:1789760715.037458:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696306502864 time 696352760709 state 0x0 ref 1 00000100:00100000:3.0:1789760715.037470:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760715.037525:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17485824, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:3.0:1789760715.037534:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.037549:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760715.037554:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.037560:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.037573:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760715.038208:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760715.064607:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971589 is committed 00002000:00010000:0.0:1789760715.064943:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17485824 -> 17489920 00010000:00010000:0.0:1789760715.064958:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.064970:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.064981:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.064997:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760715.065002:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.065010:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.065092:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.065103:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b70f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.065196:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27723us (27952us total) trans 4294971589 rc 0/0 00000100:00100000:0.0:1789760715.065207:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4299 00000100:00100000:0.0:1789760715.065217:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696306502864 time 696352760709 state 0x2 ref 0 00000100:00100000:0.0:1789760715.071824:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.072001:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174144 00000100:00100000:1.0:1789760715.072011:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174144 00000100:00100000:1.0:1789760715.072063:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696306502864 time 696352760709 state 0x0 ref 1 00000100:00100000:1.0:1789760715.072078:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696353021125 time 696352760709 state 0x1 ref 1 00000100:00100000:1.0:1789760715.072109:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696353021125 time 696387412520 state 0x0 ref 1 00000100:00100000:1.0:1789760715.072122:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.072167:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17489920, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.072176:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.072191:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.072195:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.072201:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.072214:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.105731:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971590 is committed 00002000:00010000:1.0:1789760715.105843:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17489920 -> 17494016 00010000:00010000:1.0:1789760715.105858:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105870:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105885:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105892:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760715.105897:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105907:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105926:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.105934:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b70fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.106040:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33915us (34220us total) trans 4294971590 rc 0/0 00000100:00100000:1.0:1789760715.106051:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4300 00000100:00100000:1.0:1789760715.106060:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696353021125 time 696387412520 state 0x2 ref 0 00000100:00100000:1.0:1789760715.120292:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760715.120375:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174272 00000100:00100000:3.0:1789760715.120386:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174272 00000100:00100000:3.0:1789760715.120446:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696353021125 time 696387412520 state 0x0 ref 1 00000100:00100000:3.0:1789760715.120460:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696387672936 time 696387412520 state 0x1 ref 1 00000100:00100000:3.0:1789760715.120491:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696387672936 time 696435794896 state 0x0 ref 1 00000100:00100000:3.0:1789760715.120505:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240b8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760715.120555:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17494016, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:3.0:1789760715.120563:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.120580:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760715.120595:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.120601:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.120616:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.177509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971591 is committed 00002000:00010000:1.0:1789760715.177572:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17494016 -> 17498112 00010000:00010000:1.0:1789760715.177588:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177606:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177620:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177628:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760715.177633:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177642:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177661:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.177670:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7104 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.177787:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240b8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57278us (57501us total) trans 4294971591 rc 0/0 00000100:00100000:1.0:1789760715.177796:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4301 00000100:00100000:1.0:1789760715.177806:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696387672936 time 696435794896 state 0x2 ref 0 00000100:00100000:0.0:1789760715.181767:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.181817:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174400 00000100:00100000:1.0:1789760715.181828:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174400 00000100:00100000:1.0:1789760715.181882:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696387672936 time 696435794896 state 0x0 ref 1 00000100:00100000:1.0:1789760715.181897:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696436055312 time 696435794896 state 0x1 ref 1 00000100:00100000:1.0:1789760715.181924:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696436055312 time 696497227237 state 0x0 ref 1 00000100:00100000:1.0:1789760715.181936:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.181999:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17498112, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.182009:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.182032:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.182037:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.182044:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.182059:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.208751:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17498112 -> 17502208 00010000:00010000:0.0:1789760715.208774:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.208774:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971592 is committed 00010000:00010000:0.0:1789760715.208788:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.208797:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.208805:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760715.208810:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.208819:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.208838:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.208845:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b710b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.208965:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27027us (27203us total) trans 4294971592 rc 0/0 00000100:00100000:0.0:1789760715.208973:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4302 00000100:00100000:0.0:1789760715.208982:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696436055312 time 696497227237 state 0x2 ref 0 00000100:00100000:0.0:1789760715.214384:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.214532:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174528 00000100:00100000:1.0:1789760715.214542:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174528 00000100:00100000:1.0:1789760715.214592:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696436055312 time 696497227237 state 0x0 ref 1 00000100:00100000:1.0:1789760715.214607:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696497487653 time 696497227237 state 0x1 ref 1 00000100:00100000:1.0:1789760715.214662:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696497487653 time 696529965837 state 0x0 ref 1 00000100:00100000:1.0:1789760715.214678:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.214761:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17502208, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.214770:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.214785:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.214789:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.214794:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.214808:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.264600:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971593 is committed 00002000:00010000:1.0:1789760715.264768:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17502208 -> 17506304 00010000:00010000:1.0:1789760715.264791:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264809:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264819:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264827:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760715.264833:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264844:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264870:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.264971:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7112 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17502208->17506303] (req 17502208->17506303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.265109:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50424us (50725us total) trans 4294971593 rc 0/0 00000100:00100000:1.0:1789760715.265121:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4303 00000100:00100000:1.0:1789760715.265135:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696497487653 time 696529965837 state 0x2 ref 0 00000100:00100000:1.0:1789760715.273604:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.273752:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174656 00000100:00100000:2.0:1789760715.273764:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174656 00000100:00100000:2.0:1789760715.273835:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696497487653 time 696529965837 state 0x0 ref 1 00000100:00100000:2.0:1789760715.273854:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696530226253 time 696529965837 state 0x1 ref 1 00000100:00100000:0.0:1789760715.274005:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696530226253 time 696589298530 state 0x0 ref 1 00000100:00100000:0.0:1789760715.274025:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760715.274092:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17506304, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:0.0:1789760715.274104:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.274123:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760715.274129:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.274137:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.274153:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.344465:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17506304 -> 17510400 00010000:00010000:2.0:1789760715.344495:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344512:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344523:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344531:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.344539:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344549:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344575:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.344583:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7119 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.344767:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507174656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70739us (71169us total) trans 4294971594 rc 0/0 00000100:00100000:2.0:1789760715.344781:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4304 00000100:00100000:2.0:1789760715.344793:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696530226253 time 696589298530 state 0x2 ref 0 00000001:00080000:0.0:1789760715.345382:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971594 is committed 00000100:00100000:0.0:1789760715.349457:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.349747:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174784 00000100:00100000:1.0:1789760715.349761:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174784 00000100:00100000:1.0:1789760715.349842:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696530226253 time 696589298530 state 0x0 ref 1 00000100:00100000:1.0:1789760715.349861:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696589558946 time 696589298530 state 0x1 ref 1 00000100:00100000:1.0:1789760715.350012:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696589558946 time 696665311053 state 0x0 ref 1 00000100:00100000:1.0:1789760715.350040:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507174784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.350113:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17510400, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.350125:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.350148:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.350154:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.350161:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.350179:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.406279:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17510400 -> 17514496 00010000:00010000:0.0:1789760715.406308:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406326:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406337:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406346:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760715.406352:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406363:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406389:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.406398:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7120 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.406667:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507174784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56626us (57217us total) trans 4294971595 rc 0/0 00000100:00100000:0.0:1789760715.406680:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4305 00000100:00100000:0.0:1789760715.406734:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696589558946 time 696665311053 state 0x2 ref 0 00000001:00080000:0.0:1789760715.406785:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971595 is committed 00000100:00100000:0.0:1789760715.410855:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.411080:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507174912 00000100:00100000:1.0:1789760715.411095:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507174912 00000100:00100000:1.0:1789760715.411169:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696589558946 time 696665311053 state 0x0 ref 1 00000100:00100000:1.0:1789760715.411188:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696665571469 time 696665311053 state 0x1 ref 1 00000100:00100000:1.0:1789760715.411294:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696665571469 time 696726595488 state 0x0 ref 1 00000100:00100000:1.0:1789760715.411309:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.411375:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17514496, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.411386:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.411406:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.411411:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.411419:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.411447:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.470656:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17514496 -> 17518592 00010000:00010000:2.0:1789760715.470711:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470727:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470736:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470749:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.470755:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470764:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470788:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.470796:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7127 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17514496->17518591] (req 17514496->17518591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.470966:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971596 is committed 00000100:00100000:2.0:1789760715.471240:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507174912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59926us (60393us total) trans 4294971596 rc 0/0 00000100:00100000:2.0:1789760715.471254:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4306 00000100:00100000:2.0:1789760715.471267:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696665571469 time 696726595488 state 0x2 ref 0 00000100:00100000:2.0:1789760715.476310:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760715.476414:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175040 00000100:00100000:0.0:1789760715.476437:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175040 00000100:00100000:0.0:1789760715.476646:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696665571469 time 696726595488 state 0x0 ref 1 00000100:00100000:0.0:1789760715.476664:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696726855904 time 696726595488 state 0x1 ref 1 00000100:00100000:0.0:1789760715.476725:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696726855904 time 696792027295 state 0x0 ref 1 00000100:00100000:0.0:1789760715.476740:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760715.476809:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17518592, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:0.0:1789760715.476820:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.476839:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760715.476844:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.476851:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.476867:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.533039:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971597 is committed 00002000:00010000:0.0:1789760715.533189:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17518592 -> 17522688 00010000:00010000:0.0:1789760715.533212:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533228:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533240:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533249:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760715.533254:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533264:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533290:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.533299:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b712e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17518592->17522687] (req 17518592->17522687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.533418:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56674us (57120us total) trans 4294971597 rc 0/0 00000100:00100000:0.0:1789760715.533439:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4307 00000100:00100000:0.0:1789760715.533452:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696726855904 time 696792027295 state 0x2 ref 0 00000100:00100000:0.0:1789760715.537304:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.537580:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175168 00000100:00100000:1.0:1789760715.537593:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175168 00000100:00100000:1.0:1789760715.537673:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696726855904 time 696792027295 state 0x0 ref 1 00000100:00100000:1.0:1789760715.537735:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696792287711 time 696792027295 state 0x1 ref 1 00000100:00100000:1.0:1789760715.537777:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696792287711 time 696853079464 state 0x0 ref 1 00000100:00100000:1.0:1789760715.537792:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507175168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.537861:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17522688, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.537874:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.537895:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760715.537900:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.537908:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.537926:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.592787:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971598 is committed 00002000:00010000:1.0:1789760715.592944:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17522688 -> 17526784 00010000:00010000:1.0:1789760715.592966:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.592990:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.593001:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.593009:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760715.593015:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.593092:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.593119:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.593128:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7135 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17522688->17526783] (req 17522688->17526783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.593267:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507175168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55470us (55969us total) trans 4294971598 rc 0/0 00000100:00100000:1.0:1789760715.593283:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4308 00000100:00100000:1.0:1789760715.593295:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696792287711 time 696853079464 state 0x2 ref 0 00000100:00100000:1.0:1789760715.596640:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.596733:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175296 00000100:00100000:2.0:1789760715.596747:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175296 00000100:00100000:2.0:1789760715.596821:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696792287711 time 696853079464 state 0x0 ref 1 00000100:00100000:2.0:1789760715.596840:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696853339880 time 696853079464 state 0x1 ref 1 00000100:00100000:2.0:1789760715.596877:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696853339880 time 696912179796 state 0x0 ref 1 00000100:00100000:2.0:1789760715.596893:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507175296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760715.596961:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17526784, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760715.596973:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.597002:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760715.597008:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.597015:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.597043:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.642901:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971599 is committed 00002000:00010000:2.0:1789760715.643122:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17526784 -> 17530880 00010000:00010000:2.0:1789760715.643144:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643159:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643169:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643178:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.643184:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643194:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643219:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.643228:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b713c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.643358:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507175296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46461us (46727us total) trans 4294971599 rc 0/0 00000100:00100000:2.0:1789760715.643371:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4309 00000100:00100000:2.0:1789760715.643382:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696853339880 time 696912179796 state 0x2 ref 0 00000100:00100000:0.0:1789760715.646628:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.646819:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175424 00000100:00100000:2.0:1789760715.646832:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175424 00000100:00100000:2.0:1789760715.646909:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696853339880 time 696912179796 state 0x0 ref 1 00000100:00100000:2.0:1789760715.646928:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696912440212 time 696912179796 state 0x1 ref 1 00000100:00100000:2.0:1789760715.646964:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696912440212 time 696962266889 state 0x0 ref 1 00000100:00100000:2.0:1789760715.646978:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507175424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760715.647062:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17530880, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760715.647073:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.647093:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760715.647099:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.647107:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.647125:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.676557:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760715.699796:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17530880 -> 17534976 00010000:00010000:2.0:1789760715.699824:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699840:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699850:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699858:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.699863:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699873:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699902:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.699912:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7143 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.700210:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507175424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53225us (53588us total) trans 4294971600 rc 0/0 00000100:00100000:2.0:1789760715.700223:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4310 00000100:00100000:2.0:1789760715.700235:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696912440212 time 696962266889 state 0x2 ref 0 00000001:00080000:0.0:1789760715.702893:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971600 is committed 00000100:00100000:0.0:1789760715.704559:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760715.704736:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175552 00000100:00100000:1.0:1789760715.704749:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175552 00000100:00100000:1.0:1789760715.704828:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696912440212 time 696962266889 state 0x0 ref 1 00000100:00100000:1.0:1789760715.704847:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696962527305 time 696962266889 state 0x1 ref 1 00000100:00100000:1.0:1789760715.704882:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696962527305 time 697020185633 state 0x0 ref 1 00000100:00100000:1.0:1789760715.704896:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760715.704964:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17534976, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:1.0:1789760715.704976:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.705015:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760715.705029:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.705038:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.705057:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.758897:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971601 is committed 00002000:00010000:2.0:1789760715.759143:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17534976 -> 17539072 00010000:00010000:2.0:1789760715.759165:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759182:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759192:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759200:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.759206:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759216:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759248:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.759258:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b714a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17534976->17539071] (req 17534976->17539071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.759389:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54488us (54836us total) trans 4294971601 rc 0/0 00000100:00100000:2.0:1789760715.759402:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4311 00000100:00100000:2.0:1789760715.759415:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696962527305 time 697020185633 state 0x2 ref 0 00000100:00100000:1.0:1789760715.763824:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.763947:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175680 00000100:00100000:2.0:1789760715.763959:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175680 00000100:00100000:2.0:1789760715.764052:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 696962527305 time 697020185633 state 0x0 ref 1 00000100:00100000:2.0:1789760715.764080:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697020446049 time 697020185633 state 0x1 ref 1 00000100:00100000:2.0:1789760715.764113:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697020446049 time 697079415335 state 0x0 ref 1 00000100:00100000:2.0:1789760715.764128:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760715.764195:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17539072, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760715.764206:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.764226:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760715.764232:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.764240:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.764257:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.826259:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971602 is committed 00002000:00010000:2.0:1789760715.826958:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17539072 -> 17543168 00010000:00010000:2.0:1789760715.827210:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827227:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827238:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827246:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760715.827251:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827263:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827289:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.827301:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7151 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17539072->17543167] (req 17539072->17543167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.827445:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63311us (63628us total) trans 4294971602 rc 0/0 00000100:00100000:2.0:1789760715.829587:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4312 00000100:00100000:2.0:1789760715.829609:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697020446049 time 697079415335 state 0x2 ref 0 00000100:00100000:3.0:1789760715.849008:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.852795:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175808 00000100:00100000:2.0:1789760715.852815:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175808 00000100:00100000:2.0:1789760715.852914:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697020446049 time 697079415335 state 0x0 ref 1 00000100:00100000:2.0:1789760715.852934:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697079675751 time 697079415335 state 0x1 ref 1 00000100:00100000:2.0:1789760715.852998:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697079675751 time 697168300564 state 0x0 ref 1 00000100:00100000:2.0:1789760715.853017:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507175808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760715.853118:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17543168, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760715.853130:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.853151:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760715.853157:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.853167:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.853192:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.965739:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971603 is committed 00002000:00010000:0.0:1789760715.966160:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17543168 -> 17547264 00010000:00010000:0.0:1789760715.966182:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966198:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966210:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966218:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760715.966223:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966233:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966261:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.966270:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7158 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.967527:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507175808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 113440us (118054us total) trans 4294971603 rc 0/0 00000100:00100000:0.0:1789760715.967553:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4313 00000100:00100000:0.0:1789760715.967565:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697079675751 time 697168300564 state 0x2 ref 0 00000100:00100000:0.0:1789760715.980905:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760715.981032:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507175936 00000100:00100000:2.0:1789760715.981045:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507175936 00000100:00100000:2.0:1789760715.981121:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697079675751 time 697168300564 state 0x0 ref 1 00000100:00100000:2.0:1789760715.981140:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697168560980 time 697168300564 state 0x1 ref 1 00000100:00100000:2.0:1789760715.981174:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697168560980 time 697296477158 state 0x0 ref 1 00000100:00100000:2.0:1789760715.981189:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760715.981253:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17547264, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760715.981264:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.981283:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760715.981288:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.981297:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.981313:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.029784:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971604 is committed 00002000:00010000:0.0:1789760716.029948:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17547264 -> 17551360 00010000:00010000:0.0:1789760716.029970:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.029996:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.030007:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.030188:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760716.030194:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.030207:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.030234:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.030243:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b715f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17547264->17551359] (req 17547264->17551359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.030372:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507175936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49180us (49478us total) trans 4294971604 rc 0/0 00000100:00100000:0.0:1789760716.030385:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4314 00000100:00100000:0.0:1789760716.030398:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697168560980 time 697296477158 state 0x2 ref 0 00000100:00100000:1.0:1789760716.034510:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760716.034669:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176064 00000100:00100000:2.0:1789760716.034683:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176064 00000100:00100000:2.0:1789760716.034781:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697168560980 time 697296477158 state 0x0 ref 1 00000100:00100000:2.0:1789760716.034801:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697296737574 time 697296477158 state 0x1 ref 1 00000100:00100000:2.0:1789760716.034837:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697296737574 time 697350140081 state 0x0 ref 1 00000100:00100000:2.0:1789760716.034851:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507176064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760716.034916:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17551360, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x8807 00010000:00010000:2.0:1789760716.034928:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.034949:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760716.034954:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.034962:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.035182:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.088618:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971605 is committed 00002000:00010000:3.0:1789760716.089046:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17551360 -> 17555456 00002000:00010000:3.0:1789760716.089054:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:3.0:1789760716.089057:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:3.0:1789760716.089193:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089210:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089225:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089233:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760716.089238:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089248:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089275:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.089285:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7166 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17551360->17555455] (req 17551360->17555455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.089430:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507176064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54565us (54918us total) trans 4294971605 rc 0/0 00000100:00100000:3.0:1789760716.089443:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4315 00000100:00100000:3.0:1789760716.089455:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697296737574 time 697350140081 state 0x2 ref 0 00000100:00100000:3.0:1789760716.093549:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760716.093663:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176192 00000100:00100000:0.0:1789760716.093676:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176192 00000100:00100000:0.0:1789760716.093817:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697296737574 time 697350140081 state 0x0 ref 1 00000100:00100000:0.0:1789760716.093837:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697350400497 time 697350140081 state 0x1 ref 1 00000100:00100000:0.0:1789760716.093903:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697350400497 time 697409205350 state 0x0 ref 1 00000100:00100000:0.0:1789760716.093919:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760716.093992:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17555456, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:0.0:1789760716.094003:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.094142:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760716.094148:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.094156:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.094172:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.161772:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971606 is committed 00002000:00010000:0.0:1789760716.161899:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17555456 -> 17559552 00010000:00010000:0.0:1789760716.161920:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.161935:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.161946:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.161954:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760716.161960:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.161971:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.162006:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.162159:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b716d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17555456->17559551] (req 17555456->17559551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.162295:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68372us (68874us total) trans 4294971606 rc 0/0 00000100:00100000:0.0:1789760716.162312:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4316 00000100:00100000:0.0:1789760716.162323:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697350400497 time 697409205350 state 0x2 ref 0 00000100:00100000:0.0:1789760716.168379:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760716.168865:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176320 00000100:00100000:1.0:1789760716.168879:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176320 00000100:00100000:1.0:1789760716.168956:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697350400497 time 697409205350 state 0x0 ref 1 00000100:00100000:1.0:1789760716.168988:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697409465766 time 697409205350 state 0x1 ref 1 00000100:00100000:1.0:1789760716.169031:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697409465766 time 697484333612 state 0x0 ref 1 00000100:00100000:1.0:1789760716.169045:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507176320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760716.169123:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17559552, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:1.0:1789760716.169134:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.169156:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760716.169162:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.169170:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.169188:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760716.334268:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17559552 -> 17563648 00010000:00010000:1.0:1789760716.334309:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334325:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334336:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334344:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760716.334350:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334361:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334386:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.334393:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7174 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17559552->17563647] (req 17559552->17563647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.334527:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507176320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 165476us (166156us total) trans 4294971607 rc 0/0 00000100:00100000:1.0:1789760716.334543:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4317 00000100:00100000:1.0:1789760716.334555:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697409465766 time 697484333612 state 0x2 ref 0 00000001:00080000:2.0:1789760716.342356:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971607 is committed 00000100:00100000:2.0:1789760716.343400:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760716.343500:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176448 00000100:00100000:3.0:1789760716.343513:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176448 00000100:00100000:3.0:1789760716.343589:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697409465766 time 697484333612 state 0x0 ref 1 00000100:00100000:3.0:1789760716.343612:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697484594028 time 697484333612 state 0x1 ref 1 00000100:00100000:3.0:1789760716.343650:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697484594028 time 697658953299 state 0x0 ref 1 00000100:00100000:3.0:1789760716.343665:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760716.343764:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17563648, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:3.0:1789760716.343776:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.343798:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760716.343804:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.343812:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.343830:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.397545:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971608 is committed 00002000:00010000:3.0:1789760716.397770:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17563648 -> 17567744 00010000:00010000:3.0:1789760716.397792:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397806:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397817:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397830:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760716.397836:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397847:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397871:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.397879:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b717b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17563648->17567743] (req 17563648->17567743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.398010:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54340us (54616us total) trans 4294971608 rc 0/0 00000100:00100000:3.0:1789760716.398039:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4318 00000100:00100000:3.0:1789760716.398052:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697484594028 time 697658953299 state 0x2 ref 0 00000100:00100000:3.0:1789760716.402205:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760716.402377:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176576 00000100:00100000:0.0:1789760716.402390:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176576 00000100:00100000:0.0:1789760716.402471:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697484594028 time 697658953299 state 0x0 ref 1 00000100:00100000:0.0:1789760716.402491:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697659213715 time 697658953299 state 0x1 ref 1 00000100:00100000:0.0:1789760716.402526:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697659213715 time 697717829399 state 0x0 ref 1 00000100:00100000:0.0:1789760716.402541:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760716.402607:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17567744, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:0.0:1789760716.402618:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.402636:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760716.402642:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.402650:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.402666:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.465429:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971609 is committed 00002000:00010000:2.0:1789760716.465623:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17567744 -> 17571840 00010000:00010000:2.0:1789760716.465643:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465665:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465675:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465683:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760716.465688:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465726:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465753:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.465762:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7182 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17567744->17571839] (req 17567744->17571839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.466859:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64306us (64656us total) trans 4294971609 rc 0/0 00000100:00100000:2.0:1789760716.466876:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4319 00000100:00100000:2.0:1789760716.466889:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697659213715 time 697717829399 state 0x2 ref 0 00000100:00100000:2.0:1789760716.476256:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760716.477108:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176704 00000100:00100000:1.0:1789760716.477125:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176704 00000100:00100000:1.0:1789760716.477210:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697659213715 time 697717829399 state 0x0 ref 1 00000100:00100000:1.0:1789760716.477231:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697718089815 time 697717829399 state 0x1 ref 1 00000100:00100000:1.0:1789760716.477264:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697718089815 time 697792566407 state 0x0 ref 1 00000100:00100000:1.0:1789760716.477279:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760716.477350:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17571840, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:1.0:1789760716.477360:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.477380:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760716.477385:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.477393:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.477411:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.519270:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971610 is committed 00002000:00010000:2.0:1789760716.519435:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17571840 -> 17575936 00010000:00010000:2.0:1789760716.519456:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519473:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519482:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519490:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760716.519495:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519506:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519533:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.519541:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7189 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17571840->17575935] (req 17571840->17575935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.519671:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507176704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42387us (43422us total) trans 4294971610 rc 0/0 00000100:00100000:2.0:1789760716.519683:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4320 00000100:00100000:2.0:1789760716.519727:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697718089815 time 697792566407 state 0x2 ref 0 00000100:00100000:1.0:1789760716.524108:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760716.524241:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176832 00000100:00100000:3.0:1789760716.524512:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176832 00000100:00100000:3.0:1789760716.524587:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697718089815 time 697792566407 state 0x0 ref 1 00000100:00100000:3.0:1789760716.524606:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697792826823 time 697792566407 state 0x1 ref 1 00000100:00100000:3.0:1789760716.524690:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697792826823 time 697839992452 state 0x0 ref 1 00000100:00100000:3.0:1789760716.524824:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507176832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760716.524890:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17575936, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:3.0:1789760716.524901:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.524922:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760716.524927:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.524936:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.524953:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.567941:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17575936 -> 17580032 00010000:00010000:3.0:1789760716.567970:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.567996:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.568007:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.568026:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760716.568032:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.568043:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.568067:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.568076:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7190 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17575936->17580031] (req 17575936->17580031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.568196:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507176832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43369us (44096us total) trans 4294971611 rc 0/0 00000100:00100000:3.0:1789760716.568209:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4321 00000100:00100000:3.0:1789760716.568221:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697792826823 time 697839992452 state 0x2 ref 0 00000100:00100000:0.0:1789760716.571288:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760716.571402:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507176960 00000100:00100000:2.0:1789760716.571416:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507176960 00000100:00100000:2.0:1789760716.571490:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697792826823 time 697839992452 state 0x0 ref 1 00000100:00100000:2.0:1789760716.571509:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697840252868 time 697839992452 state 0x1 ref 1 00000100:00100000:2.0:1789760716.571546:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697840252868 time 697886849196 state 0x0 ref 1 00000100:00100000:2.0:1789760716.571561:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507176960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760716.571629:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17580032, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:2.0:1789760716.571640:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.571660:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760716.571665:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.571673:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.571689:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.573191:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971611 is committed 00000001:00080000:0.0:1789760716.615068:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971612 is committed 00002000:00010000:1.0:1789760716.615140:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17580032 -> 17584128 00010000:00010000:1.0:1789760716.615161:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615177:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615188:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615208:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760716.615213:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615224:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615258:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.615267:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7197 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17580032->17584127] (req 17580032->17584127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.615394:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507176960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43829us (44115us total) trans 4294971612 rc 0/0 00000100:00100000:1.0:1789760716.615407:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4322 00000100:00100000:1.0:1789760716.615420:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697840252868 time 697886849196 state 0x2 ref 0 00000100:00100000:2.0:1789760716.618960:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760716.619093:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177088 00000100:00100000:1.0:1789760716.619106:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177088 00000100:00100000:1.0:1789760716.619178:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697840252868 time 697886849196 state 0x0 ref 1 00000100:00100000:1.0:1789760716.619198:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697887109612 time 697886849196 state 0x1 ref 1 00000100:00100000:1.0:1789760716.619236:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697887109612 time 697934539003 state 0x0 ref 1 00000100:00100000:1.0:1789760716.619260:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760716.619329:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17584128, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:1.0:1789760716.619340:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.619359:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760716.619365:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.619372:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.619389:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.671262:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17584128 -> 17588224 00010000:00010000:2.0:1789760716.671291:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671315:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671328:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671337:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760716.671343:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671355:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671383:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.671392:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b719e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17584128->17588223] (req 17584128->17588223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.671533:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507177088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52269us (52594us total) trans 4294971613 rc 0/0 00000100:00100000:2.0:1789760716.671546:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4323 00000100:00100000:2.0:1789760716.671558:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697887109612 time 697934539003 state 0x2 ref 0 00000001:00080000:1.0:1789760716.671917:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971613 is committed 00000100:00100000:1.0:1789760716.675666:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760716.675815:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177216 00000100:00100000:2.0:1789760716.675830:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177216 00000100:00100000:2.0:1789760716.675907:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697887109612 time 697934539003 state 0x0 ref 1 00000100:00100000:2.0:1789760716.675926:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697934799419 time 697934539003 state 0x1 ref 1 00000100:00100000:2.0:1789760716.675962:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697934799419 time 697991265075 state 0x0 ref 1 00000100:00100000:2.0:1789760716.675987:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760716.676064:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17588224, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:2.0:1789760716.676075:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.676095:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760716.676101:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.676109:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.676125:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.700924:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760716.744553:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971614 is committed 00002000:00010000:2.0:1789760716.744762:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17588224 -> 17592320 00010000:00010000:2.0:1789760716.744783:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744800:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744815:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744824:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760716.744830:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744840:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744866:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.744875:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b71a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17588224->17592319] (req 17588224->17592319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.745003:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69021us (69347us total) trans 4294971614 rc 0/0 00000100:00100000:2.0:1789760716.747712:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4324 00000100:00100000:2.0:1789760716.747729:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697934799419 time 697991265075 state 0x2 ref 0 00000100:00100000:1.0:1789760716.749251:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760716.749389:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177344 00000100:00100000:2.0:1789760716.749411:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177344 00000100:00100000:2.0:1789760716.749768:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697934799419 time 697991265075 state 0x0 ref 1 00000100:00100000:2.0:1789760716.749787:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697991525491 time 697991265075 state 0x1 ref 1 00000100:00100000:2.0:1789760716.749825:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697991525491 time 698065127406 state 0x0 ref 1 00000100:00100000:2.0:1789760716.749838:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760716.749904:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17592320, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:2.0:1789760716.749916:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.749936:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760716.749941:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.749950:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.749967:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.794667:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971615 is committed 00002000:00010000:1.0:1789760716.794875:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17592320 -> 17596416 00010000:00010000:1.0:1789760716.794896:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.794912:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.794923:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.794931:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760716.794937:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.794948:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.795134:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.795146:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b71ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17592320->17596415] (req 17592320->17596415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.795397:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45553us (46150us total) trans 4294971615 rc 0/0 00000100:00100000:1.0:1789760716.795545:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4325 00000100:00100000:1.0:1789760716.795560:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697991525491 time 698065127406 state 0x2 ref 0 00000100:00100000:1.0:1789760716.800583:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760716.800886:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177472 00000100:00100000:3.0:1789760716.800899:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177472 00000100:00100000:3.0:1789760716.800985:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 697991525491 time 698065127406 state 0x0 ref 1 00000100:00100000:3.0:1789760716.801075:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698065387822 time 698065127406 state 0x1 ref 1 00000100:00100000:3.0:1789760716.801145:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698065387822 time 698116446399 state 0x0 ref 1 00000100:00100000:3.0:1789760716.801161:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507177472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760716.801321:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17596416, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:3.0:1789760716.801332:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.801351:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760716.801356:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.801363:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.801379:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.850267:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17596416 -> 17600512 00010000:00010000:0.0:1789760716.850292:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850306:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850316:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850323:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760716.850329:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850339:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850364:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.850372:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b71b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17596416->17600511] (req 17596416->17600511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.850682:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507177472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49517us (50106us total) trans 4294971616 rc 0/0 00000100:00100000:0.0:1789760716.850736:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4326 00000100:00100000:0.0:1789760716.850748:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698065387822 time 698116446399 state 0x2 ref 0 00000001:00080000:2.0:1789760716.853598:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971616 is committed 00000100:00100000:2.0:1789760716.854313:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760716.854546:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177600 00000100:00100000:0.0:1789760716.854560:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177600 00000100:00100000:0.0:1789760716.854633:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698065387822 time 698116446399 state 0x0 ref 1 00000100:00100000:0.0:1789760716.854653:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698116706815 time 698116446399 state 0x1 ref 1 00000100:00100000:0.0:1789760716.854781:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698116706815 time 698170082637 state 0x0 ref 1 00000100:00100000:0.0:1789760716.854795:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507177600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760716.854858:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17600512, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:0.0:1789760716.854869:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.854887:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760716.854892:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.854898:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.854914:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.914885:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971617 is committed 00002000:00010000:0.0:1789760716.914995:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17600512 -> 17604608 00010000:00010000:0.0:1789760716.915258:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915275:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915286:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915295:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760716.915301:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915312:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915339:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.915349:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17600512->17604607] (req 17600512->17604607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.915590:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507177600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60789us (61277us total) trans 4294971617 rc 0/0 00000100:00100000:0.0:1789760716.915604:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4327 00000100:00100000:0.0:1789760716.915616:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698116706815 time 698170082637 state 0x2 ref 0 00000100:00100000:2.0:1789760716.919687:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760716.919846:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177728 00000100:00100000:1.0:1789760716.919862:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177728 00000100:00100000:1.0:1789760716.919948:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698116706815 time 698170082637 state 0x0 ref 1 00000100:00100000:1.0:1789760716.919968:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698170343053 time 698170082637 state 0x1 ref 1 00000100:00100000:1.0:1789760716.920020:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698170343053 time 698235322409 state 0x0 ref 1 00000100:00100000:1.0:1789760716.920034:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760716.920100:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17604608, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:1.0:1789760716.920110:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.920131:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760716.920136:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.920144:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.920161:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.003561:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971618 is committed 00002000:00010000:0.0:1789760717.006335:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17604608 -> 17608704 00010000:00010000:0.0:1789760717.006363:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006389:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006399:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006422:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760717.006437:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006448:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006822:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.006833:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b71c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.006985:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507177728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 86937us (87298us total) trans 4294971618 rc 0/0 00000100:00100000:0.0:1789760717.006997:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4328 00000100:00100000:0.0:1789760717.007011:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698170343053 time 698235322409 state 0x2 ref 0 00000100:00100000:1.0:1789760717.009843:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760717.009942:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177856 00000100:00100000:0.0:1789760717.009955:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177856 00000100:00100000:0.0:1789760717.010068:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698170343053 time 698235322409 state 0x0 ref 1 00000100:00100000:0.0:1789760717.010086:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698235582825 time 698235322409 state 0x1 ref 1 00000100:00100000:0.0:1789760717.010121:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698235582825 time 698325423414 state 0x0 ref 1 00000100:00100000:0.0:1789760717.010137:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507177856:12345-192.168.203.41@tcp:4:dd.0 00000100:00080000:2.0:1789760717.010164:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760717.010210:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17608704, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x8807 00010000:00010000:0.0:1789760717.010221:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.010242:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760717.010247:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.010255:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.010272:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.075913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971619 is committed 00002000:00010000:0.0:1789760717.078070:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17608704 -> 17612800 00002000:00010000:0.0:1789760717.078081:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:0.0:1789760717.078084:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:0.0:1789760717.078101:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078118:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078129:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078138:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760717.078143:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078151:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078175:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.078182:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17608704->17612799] (req 17608704->17612799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.078319:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507177856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68179us (68483us total) trans 4294971619 rc 0/0 00000100:00100000:0.0:1789760717.078331:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4329 00000100:00100000:0.0:1789760717.078343:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698235582825 time 698325423414 state 0x2 ref 0 00000100:00100000:0.0:1789760717.082410:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760717.082569:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507177984 00000100:00100000:2.0:1789760717.082584:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507177984 00000100:00100000:2.0:1789760717.082655:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698235582825 time 698325423414 state 0x0 ref 1 00000100:00100000:2.0:1789760717.082674:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698325683830 time 698325423414 state 0x1 ref 1 00000100:00100000:2.0:1789760717.082747:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698325683830 time 698398049427 state 0x0 ref 1 00000100:00100000:2.0:1789760717.082762:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507177984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760717.082830:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17612800, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:2.0:1789760717.082840:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.082861:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760717.082867:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.082875:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.082892:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.131521:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971620 is committed 00002000:00010000:0.0:1789760717.131606:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17612800 -> 17616896 00010000:00010000:0.0:1789760717.131628:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131644:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131654:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131662:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760717.131667:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131678:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131727:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.131737:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17612800->17616895] (req 17612800->17616895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.131907:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507177984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49140us (49506us total) trans 4294971620 rc 0/0 00000100:00100000:0.0:1789760717.131921:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4330 00000100:00100000:0.0:1789760717.131934:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698325683830 time 698398049427 state 0x2 ref 0 00000100:00100000:2.0:1789760717.143952:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760717.144162:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178112 00000100:00100000:0.0:1789760717.144175:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178112 00000100:00100000:0.0:1789760717.144250:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698325683830 time 698398049427 state 0x0 ref 1 00000100:00100000:0.0:1789760717.144271:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698398309843 time 698398049427 state 0x1 ref 1 00000100:00100000:0.0:1789760717.144309:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698398309843 time 698459612302 state 0x0 ref 1 00000100:00100000:0.0:1789760717.144324:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507178112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760717.144389:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17616896, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:0.0:1789760717.144401:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.144421:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760717.144427:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.144435:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.144463:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.272101:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971621 is committed 00002000:00010000:3.0:1789760717.272370:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17616896 -> 17620992 00010000:00010000:3.0:1789760717.272392:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272481:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272491:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272500:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760717.272505:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272516:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272544:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.272553:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b71d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.272687:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507178112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 128359us (128743us total) trans 4294971621 rc 0/0 00000100:00100000:3.0:1789760717.272744:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4331 00000100:00100000:3.0:1789760717.272756:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698398309843 time 698459612302 state 0x2 ref 0 00000100:00100000:1.0:1789760717.278872:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760717.279133:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178240 00000100:00100000:3.0:1789760717.279146:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178240 00000100:00100000:3.0:1789760717.279228:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698398309843 time 698459612302 state 0x0 ref 1 00000100:00100000:3.0:1789760717.279246:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698459872718 time 698459612302 state 0x1 ref 1 00000100:00100000:3.0:1789760717.279282:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698459872718 time 698594584948 state 0x0 ref 1 00000100:00100000:3.0:1789760717.279296:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507178240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760717.279360:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17620992, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:3.0:1789760717.279371:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.279391:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760717.279498:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.279510:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.279529:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.379245:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971622 is committed 00002000:00010000:0.0:1789760717.379438:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17620992 -> 17625088 00010000:00010000:0.0:1789760717.379453:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379678:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379687:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379713:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760717.379718:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379728:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379747:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.379755:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.379894:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507178240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 100594us (101030us total) trans 4294971622 rc 0/0 00000100:00100000:0.0:1789760717.379905:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4332 00000100:00100000:0.0:1789760717.379914:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698459872718 time 698594584948 state 0x2 ref 0 00000100:00100000:0.0:1789760717.384232:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760717.384304:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178368 00000100:00100000:2.0:1789760717.384313:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178368 00000100:00100000:2.0:1789760717.384360:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698459872718 time 698594584948 state 0x0 ref 1 00000100:00100000:2.0:1789760717.384371:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698594845364 time 698594584948 state 0x1 ref 1 00000100:00100000:2.0:1789760717.384395:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698594845364 time 698699699717 state 0x0 ref 1 00000100:00100000:2.0:1789760717.384404:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507178368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760717.384534:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17625088, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:2.0:1789760717.384544:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.384559:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760717.384563:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.384633:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.384650:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.402838:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971623 is committed 00002000:00010000:3.0:1789760717.403167:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17625088 -> 17629184 00010000:00010000:3.0:1789760717.403182:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403195:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403204:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403210:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760717.403215:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403224:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403244:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.403256:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b71e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.403394:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507178368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 18985us (19167us total) trans 4294971623 rc 0/0 00000100:00100000:3.0:1789760717.403406:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4333 00000100:00100000:3.0:1789760717.403416:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698594845364 time 698699699717 state 0x2 ref 0 00000100:00100000:3.0:1789760717.406798:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760717.406894:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178496 00000100:00100000:0.0:1789760717.406903:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178496 00000100:00100000:0.0:1789760717.406955:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698594845364 time 698699699717 state 0x0 ref 1 00000100:00100000:0.0:1789760717.407273:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698699960133 time 698699699717 state 0x1 ref 1 00000100:00100000:0.0:1789760717.407313:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698699960133 time 698722616567 state 0x0 ref 1 00000100:00100000:0.0:1789760717.407330:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507178496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760717.407375:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17629184, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:0.0:1789760717.407381:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.407395:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760717.407398:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.407403:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.407413:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.466055:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971624 is committed 00002000:00010000:3.0:1789760717.467023:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17629184 -> 17633280 00010000:00010000:3.0:1789760717.467042:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467405:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467416:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467423:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760717.467427:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467437:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467463:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467470:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b71eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.467654:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507178496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60324us (60861us total) trans 4294971624 rc 0/0 00000100:00100000:3.0:1789760717.467665:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4334 00000100:00100000:3.0:1789760717.467675:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698699960133 time 698722616567 state 0x2 ref 0 00000100:00100000:1.0:1789760717.478458:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760717.478672:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178624 00000100:00100000:0.0:1789760717.478682:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178624 00000100:00100000:0.0:1789760717.478775:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698699960133 time 698722616567 state 0x0 ref 1 00000100:00100000:0.0:1789760717.478792:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698722876983 time 698722616567 state 0x1 ref 1 00000100:00100000:0.0:1789760717.478834:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698722876983 time 698794137110 state 0x0 ref 1 00000100:00100000:0.0:1789760717.478846:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507178624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760717.478893:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17633280, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:0.0:1789760717.478902:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.478918:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760717.478922:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.478929:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.478943:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.527141:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760717.527189:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760717.527229:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760717.527252:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760717.527272:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760717.527303:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178752 00000100:00100000:1.0:1789760717.527317:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178752 00000100:00100000:1.0:1789760717.527382:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 458 00000100:00100000:1.0:1789760717.527395:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223497450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507178752:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760717.527486:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223497450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507178752:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 94us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.527496:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 458 00000100:00100000:2.0:1789760717.527681:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507178880 00000100:00100000:2.0:1789760717.527686:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507178880 00000100:00100000:2.0:1789760717.527772:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179136 00000100:00100000:2.0:1789760717.527776:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179136 00000100:00100000:2.0:1789760717.527792:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179008 00000100:00100000:2.0:1789760717.527795:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179008 00000100:00100000:2.0:1789760717.527808:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179264 00000100:00100000:2.0:1789760717.527810:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179264 00000100:00100000:2.0:1789760717.527826:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 838 00000100:00100000:2.0:1789760717.527833:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507178880:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760717.527869:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507178880:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 36us (745us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760717.527875:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 838 00000100:00100000:2.0:1789760717.527889:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 839 00000100:00100000:2.0:1789760717.527895:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507179136:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760717.527920:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507179136:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (730us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760717.527926:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 839 00000100:00100000:2.0:1789760717.527936:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 840 00000100:00100000:2.0:1789760717.527943:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507179008:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760717.527979:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507179008:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (715us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760717.527986:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 840 00000100:00100000:2.0:1789760717.527998:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 841 00000100:00100000:2.0:1789760717.528005:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507179264:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760717.528032:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507179264:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 27us (758us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760717.528038:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 841 00000100:00080000:1.0:1789760717.528397:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760717.528403:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760717.574647:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971625 is committed 00002000:00010000:2.0:1789760717.574748:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17633280 -> 17637376 00010000:00010000:2.0:1789760717.574769:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574784:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574793:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574801:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760717.574807:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574817:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574841:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.574849:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b71f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17633280->17637375] (req 17633280->17637375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.574999:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507178624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96147us (96545us total) trans 4294971625 rc 0/0 00000100:00100000:2.0:1789760717.575020:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4335 00000100:00100000:2.0:1789760717.575033:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698722876983 time 698794137110 state 0x2 ref 0 00000100:00100000:1.0:1789760717.580109:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760717.580215:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179392 00000100:00100000:3.0:1789760717.580228:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179392 00000100:00100000:3.0:1789760717.580303:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698722876983 time 698794137110 state 0x0 ref 1 00000100:00100000:3.0:1789760717.580321:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698794397526 time 698794137110 state 0x1 ref 1 00000100:00100000:3.0:1789760717.580355:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698794397526 time 698895657044 state 0x0 ref 1 00000100:00100000:3.0:1789760717.580369:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507179392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760717.580437:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17637376, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:3.0:1789760717.580448:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.580469:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760717.580475:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.580482:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.580499:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.655594:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971626 is committed 00002000:00010000:2.0:1789760717.655740:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17637376 -> 17641472 00010000:00010000:2.0:1789760717.655760:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655775:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655785:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655799:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760717.655805:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655814:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655839:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.655848:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b71f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17637376->17641471] (req 17637376->17641471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.655961:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507179392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75588us (75863us total) trans 4294971626 rc 0/0 00000100:00100000:2.0:1789760717.655983:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4336 00000100:00100000:2.0:1789760717.655995:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698794397526 time 698895657044 state 0x2 ref 0 00000100:00100000:0.0:1789760717.659040:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760717.659187:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179520 00000100:00100000:3.0:1789760717.659200:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179520 00000100:00100000:3.0:1789760717.659267:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698794397526 time 698895657044 state 0x0 ref 1 00000100:00100000:3.0:1789760717.659286:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698895917460 time 698895657044 state 0x1 ref 1 00000100:00100000:3.0:1789760717.659321:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698895917460 time 698974623761 state 0x0 ref 1 00000100:00100000:3.0:1789760717.659335:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507179520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760717.659395:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17641472, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:3.0:1789760717.659406:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.659436:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760717.659441:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.659450:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.659466:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.714731:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971627 is committed 00010000:00010000:3.0:1789760717.724960:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760717.727403:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17641472 -> 17645568 00010000:00010000:2.0:1789760717.727513:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727529:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727545:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727554:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760717.727560:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727570:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727597:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727605:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7200 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.727848:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507179520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68506us (68813us total) trans 4294971627 rc 0/0 00000100:00100000:2.0:1789760717.727861:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4337 00000100:00100000:2.0:1789760717.727874:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698895917460 time 698974623761 state 0x2 ref 0 00000100:00100000:3.0:1789760717.735036:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760717.735228:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179648 00000100:00100000:2.0:1789760717.735242:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179648 00000100:00100000:2.0:1789760717.735319:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698895917460 time 698974623761 state 0x0 ref 1 00000100:00100000:2.0:1789760717.735336:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698974884177 time 698974623761 state 0x1 ref 1 00000100:00100000:2.0:1789760717.735356:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698974884177 time 699050659339 state 0x0 ref 1 00000100:00100000:2.0:1789760717.735370:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507179648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760717.735450:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17645568, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:2.0:1789760717.735464:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.735484:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760717.735490:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.735497:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.735516:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760717.808787:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17645568 -> 17649664 00010000:00010000:2.0:1789760717.808815:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808830:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808840:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808849:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760717.808855:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808866:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808888:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.808896:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7207 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.809152:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507179648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73776us (74123us total) trans 4294971628 rc 0/0 00000100:00100000:2.0:1789760717.809166:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4338 00000100:00100000:2.0:1789760717.809179:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698974884177 time 699050659339 state 0x2 ref 0 00000001:00080000:3.0:1789760717.809356:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971628 is committed 00000100:00100000:1.0:1789760717.814248:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760717.814451:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179776 00000100:00100000:2.0:1789760717.814465:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179776 00000100:00100000:2.0:1789760717.814613:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 698974884177 time 699050659339 state 0x0 ref 1 00000100:00100000:2.0:1789760717.814632:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699050919755 time 699050659339 state 0x1 ref 1 00000100:00100000:2.0:1789760717.814669:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699050919755 time 699129971219 state 0x0 ref 1 00000100:00100000:2.0:1789760717.814683:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507179776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760717.814791:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17649664, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:2.0:1789760717.814802:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.814821:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760717.814826:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.814834:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.814850:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.917345:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971629 is committed 00002000:00010000:2.0:1789760717.920656:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17649664 -> 17653760 00010000:00010000:2.0:1789760717.920686:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920715:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920725:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920733:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760717.920738:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920748:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920780:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.920788:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b720e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17649664->17653759] (req 17649664->17653759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.920920:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507179776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 106233us (106679us total) trans 4294971629 rc 0/0 00000100:00100000:2.0:1789760717.920931:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4339 00000100:00100000:2.0:1789760717.920942:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699050919755 time 699129971219 state 0x2 ref 0 00000100:00100000:0.0:1789760717.929147:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760717.929269:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507179904 00000100:00100000:2.0:1789760717.929282:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507179904 00000100:00100000:2.0:1789760717.929359:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699050919755 time 699129971219 state 0x0 ref 1 00000100:00100000:2.0:1789760717.929380:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699130231635 time 699129971219 state 0x1 ref 1 00000100:00100000:1.0:1789760717.929675:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699130231635 time 699244974935 state 0x0 ref 1 00000100:00100000:1.0:1789760717.929742:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507179904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760717.929811:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17653760, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:1.0:1789760717.929822:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.929841:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760717.929847:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.929854:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.929880:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.028036:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971630 is committed 00002000:00010000:3.0:1789760718.028212:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17653760 -> 17657856 00010000:00010000:3.0:1789760718.028690:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028743:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028753:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028762:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.028767:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028779:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028803:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.028812:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7215 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.028939:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507179904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 99244us (99801us total) trans 4294971630 rc 0/0 00000100:00100000:3.0:1789760718.028952:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4340 00000100:00100000:3.0:1789760718.028975:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699130231635 time 699244974935 state 0x2 ref 0 00000100:00100000:0.0:1789760718.033099:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.033228:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180032 00000100:00100000:2.0:1789760718.033241:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180032 00000100:00100000:2.0:1789760718.033310:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699130231635 time 699244974935 state 0x0 ref 1 00000100:00100000:2.0:1789760718.033328:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699245235351 time 699244974935 state 0x1 ref 1 00000100:00100000:2.0:1789760718.033357:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699245235351 time 699348659882 state 0x0 ref 1 00000100:00100000:2.0:1789760718.033369:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507180032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.033448:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17657856, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x8807 00010000:00010000:2.0:1789760718.033460:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.033480:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.033486:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.033493:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.033509:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.046051:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538262016:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760718.046145:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.052679:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538262144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.052802:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.052877:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538262656:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.053044:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262144 00000100:00100000:1.0:1789760718.053054:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262144 00000100:00100000:1.0:1789760718.053118:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:1.0:1789760718.053129:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223497850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538262144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.053194:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223497850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538262144:12345-0@lo:400:kworker.0 Request processed in 66us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.053203:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1789760718.053478:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.053518:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538263168:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.053783:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538262400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.053820:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.053844:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538262912:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.053872:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.053891:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538263424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.053927:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.053952:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:0.0:1789760718.053959:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538262272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.053996:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.054024:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538262784:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.054050:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.054069:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538263296:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.054091:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.054111:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538263808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760718.054136:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.054176:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262912 00000100:00100000:0.0:1789760718.054182:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262912 00000100:00100000:0.0:1789760718.054242:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263424 00000100:00100000:0.0:1789760718.054245:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263424 00000100:00100000:0.0:1789760718.054263:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262272 00000100:00100000:0.0:1789760718.054265:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262272 00000100:00100000:0.0:1789760718.054277:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262784 00000100:00100000:0.0:1789760718.054279:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262784 00000100:00100000:0.0:1789760718.054291:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263296 00000100:00100000:0.0:1789760718.054293:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263296 00000100:00100000:0.0:1789760718.054308:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 842 00000100:00100000:0.0:1789760718.054316:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538262912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.054364:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538262912:12345-0@lo:400:kworker.0 Request processed in 48us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760718.054371:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 842 00000100:00100000:0.0:1789760718.054387:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 843 00000100:00100000:0.0:1789760718.054393:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538263424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.054788:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760718.054955:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 844 00000100:00100000:2.0:1789760718.054978:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538262272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055090:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538263680:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.055126:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262656 00000100:00100000:3.0:1789760718.055126:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760718.055132:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262656 00000100:00100000:1.0:1789760718.055165:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262400 00000100:00100000:1.0:1789760718.055168:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262400 00000100:00100000:3.0:1789760718.055169:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538262144:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.055181:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789760718.055187:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538262656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055204:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262016 00000100:00100000:3.0:1789760718.055207:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262016 00000100:00100000:1.0:1789760718.055231:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538262656:12345-0@lo:400:kworker.0 Request processed in 43us (1752us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.055237:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:3.0:1789760718.055250:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263808 00000100:00100000:1.0:1789760718.055251:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:3.0:1789760718.055253:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263808 00000100:00100000:1.0:1789760718.055256:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538262400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055268:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1789760718.055273:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538262016:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760718.055285:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538262400:12345-0@lo:400:kworker.0 Request processed in 29us (1465us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.055290:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00080000:1.0:1789760718.055306:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00000024:3.0:1789760718.055306:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00080000:1.0:1789760718.055312:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760718.055314:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760718.055343:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538262016:12345-0@lo:13:osp-pre-1-1.0 Request processed in 70us (9199us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055350:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1789760718.055366:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789760718.055372:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b458380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538263808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760718.055382:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760718.055407:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b458380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538263808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (1270us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055422:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789760718.055449:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 845 00000100:00100000:3.0:1789760718.055457:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538262784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055490:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538262784:12345-0@lo:400:kworker.0 Request processed in 35us (1439us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055497:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 845 00000100:00100000:3.0:1789760718.055511:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 846 00000100:00100000:3.0:1789760718.055518:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538263296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055661:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538263296:12345-0@lo:400:kworker.0 Request processed in 144us (1569us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055668:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 846 00000100:00100000:3.0:1789760718.055688:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263168 00000100:00100000:3.0:1789760718.055723:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263168 00000100:00100000:3.0:1789760718.055753:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263680 00000100:00100000:3.0:1789760718.055756:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263680 00000100:00100000:3.0:1789760718.055770:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1789760718.055775:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538263168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055809:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538263168:12345-0@lo:400:kworker.0 Request processed in 33us (1022us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055815:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1789760718.055826:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1789760718.055831:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538263680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055854:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538263680:12345-0@lo:400:kworker.0 Request processed in 23us (727us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.055859:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1789760718.055903:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538262656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055915:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538263168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055924:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538263680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.055972:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538262016:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760718.055981:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538262528:0@lo:400:kworker.0 00000100:00080000:1.0:1789760718.056060:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760718.056130:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.056164:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538263040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.056188:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.056207:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538263552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.056229:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.056263:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263040 00000100:00100000:3.0:1789760718.056268:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263040 00000100:00100000:3.0:1789760718.056296:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263552 00000100:00100000:3.0:1789760718.056300:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263552 00000100:00100000:3.0:1789760718.056315:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:3.0:1789760718.056321:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538263040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.056358:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538263040:12345-0@lo:400:kworker.0 Request processed in 36us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.056364:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:3.0:1789760718.056375:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789760718.056380:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538263552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.056506:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538262528 00000100:00100000:1.0:1789760718.056511:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538262528 00000100:00100000:1.0:1789760718.056532:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 847 00000100:00100000:1.0:1789760718.056538:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538262528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.056724:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538263552:12345-0@lo:400:kworker.0 Request processed in 342us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.056730:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00080000:3.0:1789760718.056748:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760718.056773:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538263040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.056782:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538263552:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.057252:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538262528:12345-0@lo:400:kworker.0 Request processed in 711us (1120us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.057260:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 847 00000100:00100000:3.0:1789760718.057351:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538262528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760718.063932:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538262272:12345-0@lo:400:kworker.0 Request processed in 8954us (9921us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760718.063949:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 844 00000100:00100000:0.0:1789760718.076937:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538263424:12345-0@lo:400:kworker.0 Request processed in 22529us (22994us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760718.076956:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 843 00000100:00100000:0.0:1789760718.077071:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538262272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.077087:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538262784:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.077100:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538263296:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.077128:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538263808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760718.077143:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538262400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.077156:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538262912:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.077168:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538263424:0@lo:400:kworker.0 00000001:00080000:2.0:1789760718.132128:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971631 is committed 00002000:00010000:2.0:1789760718.132251:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17657856 -> 17661952 00002000:00010000:2.0:1789760718.132256:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:2.0:1789760718.132259:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:2.0:1789760718.132277:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132294:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132310:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132318:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.132323:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132334:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132359:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.132368:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b721c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.132481:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507180032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 99106us (99387us total) trans 4294971631 rc 0/0 00000100:00100000:2.0:1789760718.132493:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4341 00000100:00100000:2.0:1789760718.132504:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699245235351 time 699348659882 state 0x2 ref 0 00000100:00100000:0.0:1789760718.137191:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.137293:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180160 00000100:00100000:2.0:1789760718.137307:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180160 00000100:00100000:2.0:1789760718.137387:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699245235351 time 699348659882 state 0x0 ref 1 00000100:00100000:2.0:1789760718.137405:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699348920298 time 699348659882 state 0x1 ref 1 00000100:00100000:2.0:1789760718.137441:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699348920298 time 699452743711 state 0x0 ref 1 00000100:00100000:2.0:1789760718.137456:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.137535:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17661952, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.137546:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.137565:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.137570:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.137579:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.137596:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.197384:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971632 is committed 00002000:00010000:2.0:1789760718.197479:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17661952 -> 17666048 00010000:00010000:2.0:1789760718.197500:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197514:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197524:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197538:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.197544:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197553:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197578:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.197587:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7223 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17661952->17666047] (req 17661952->17666047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.197743:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60241us (60519us total) trans 4294971632 rc 0/0 00000100:00100000:2.0:1789760718.197756:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4342 00000100:00100000:2.0:1789760718.197769:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699348920298 time 699452743711 state 0x2 ref 0 00000100:00100000:1.0:1789760718.211408:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.214188:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180288 00000100:00100000:3.0:1789760718.214212:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180288 00000100:00100000:2.0:1789760718.216484:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699348920298 time 699452743711 state 0x0 ref 1 00000100:00100000:2.0:1789760718.216511:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699453004127 time 699452743711 state 0x1 ref 1 00000100:00100000:2.0:1789760718.216611:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699453004127 time 699531912416 state 0x0 ref 1 00000100:00100000:2.0:1789760718.216631:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507180288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.216719:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17666048, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.216731:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216751:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.216756:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.216763:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216780:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.278401:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971633 is committed 00002000:00010000:2.0:1789760718.278502:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17666048 -> 17670144 00010000:00010000:2.0:1789760718.278524:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278540:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278550:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278559:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.278564:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278582:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278607:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.278617:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b722a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17666048->17670143] (req 17666048->17670143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.278779:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507180288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62148us (67562us total) trans 4294971633 rc 0/0 00000100:00100000:2.0:1789760718.278791:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4343 00000100:00100000:2.0:1789760718.278803:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699453004127 time 699531912416 state 0x2 ref 0 00000100:00100000:2.0:1789760718.288679:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760718.292113:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180416 00000100:00100000:0.0:1789760718.292134:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180416 00000100:00100000:1.0:1789760718.292464:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699453004127 time 699531912416 state 0x0 ref 1 00000100:00100000:1.0:1789760718.292487:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699532172832 time 699531912416 state 0x1 ref 1 00000100:00100000:1.0:1789760718.292526:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699532172832 time 699607828633 state 0x0 ref 1 00000100:00100000:1.0:1789760718.292543:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760718.292621:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17670144, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:1.0:1789760718.292632:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.292654:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760718.292661:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.292670:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.292690:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.360822:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971634 is committed 00002000:00010000:1.0:1789760718.360926:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17670144 -> 17674240 00010000:00010000:1.0:1789760718.360945:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.360969:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.360979:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.360987:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760718.360993:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.361067:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.361093:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.361101:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7231 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.361285:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68738us (72611us total) trans 4294971634 rc 0/0 00000100:00100000:1.0:1789760718.361301:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4344 00000100:00100000:1.0:1789760718.361314:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699532172832 time 699607828633 state 0x2 ref 0 00000100:00100000:0.0:1789760718.365436:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.365604:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180544 00000100:00100000:2.0:1789760718.365618:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180544 00000100:00100000:2.0:1789760718.365689:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699532172832 time 699607828633 state 0x0 ref 1 00000100:00100000:2.0:1789760718.365749:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699608089049 time 699607828633 state 0x1 ref 1 00000100:00100000:2.0:1789760718.365772:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699608089049 time 699681075593 state 0x0 ref 1 00000100:00100000:2.0:1789760718.365784:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.365850:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17674240, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.365861:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.365883:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.365888:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.365895:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.365912:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.401977:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971635 is committed 00002000:00010000:3.0:1789760718.402297:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17674240 -> 17678336 00010000:00010000:3.0:1789760718.402318:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402333:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402343:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402352:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.402358:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402374:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402402:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.402411:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7238 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17674240->17678335] (req 17674240->17678335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.402656:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36865us (37228us total) trans 4294971635 rc 0/0 00000100:00100000:3.0:1789760718.402669:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4345 00000100:00100000:3.0:1789760718.402682:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699608089049 time 699681075593 state 0x2 ref 0 00000100:00100000:1.0:1789760718.407456:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.407637:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180672 00000100:00100000:2.0:1789760718.407651:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180672 00000100:00100000:2.0:1789760718.407763:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699608089049 time 699681075593 state 0x0 ref 1 00000100:00100000:2.0:1789760718.407782:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699681336009 time 699681075593 state 0x1 ref 1 00000100:00100000:2.0:1789760718.407807:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699681336009 time 699723110399 state 0x0 ref 1 00000100:00100000:2.0:1789760718.407822:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507180672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.407888:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17678336, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.407898:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.407918:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.407925:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.407933:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.407949:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.441016:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971636 is committed 00002000:00010000:2.0:1789760718.441200:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17678336 -> 17682432 00010000:00010000:2.0:1789760718.441220:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441237:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441247:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441255:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.441261:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441271:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441296:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.441303:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b723f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17678336->17682431] (req 17678336->17682431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.441434:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507180672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33608us (33988us total) trans 4294971636 rc 0/0 00000100:00100000:2.0:1789760718.441447:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4346 00000100:00100000:2.0:1789760718.441574:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699681336009 time 699723110399 state 0x2 ref 0 00000100:00100000:2.0:1789760718.445823:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760718.445939:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180800 00000100:00100000:1.0:1789760718.445952:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180800 00000100:00100000:1.0:1789760718.446089:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699681336009 time 699723110399 state 0x0 ref 1 00000100:00100000:1.0:1789760718.446109:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699723370815 time 699723110399 state 0x1 ref 1 00000100:00100000:1.0:1789760718.446196:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699723370815 time 699761497345 state 0x0 ref 1 00000100:00100000:1.0:1789760718.446212:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760718.446279:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17682432, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:1.0:1789760718.446289:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.446310:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760718.446315:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.446324:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.446341:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.485242:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971637 is committed 00002000:00010000:3.0:1789760718.485383:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17682432 -> 17686528 00010000:00010000:3.0:1789760718.485403:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485426:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485436:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485448:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.485453:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485470:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485495:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485503:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7246 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.485631:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507180800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39414us (39815us total) trans 4294971637 rc 0/0 00000100:00100000:3.0:1789760718.485643:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4347 00000100:00100000:3.0:1789760718.485656:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699723370815 time 699761497345 state 0x2 ref 0 00000100:00100000:0.0:1789760718.488953:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.489083:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507180928 00000100:00100000:3.0:1789760718.489096:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507180928 00000100:00100000:3.0:1789760718.489166:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699723370815 time 699761497345 state 0x0 ref 1 00000100:00100000:3.0:1789760718.489184:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699761757761 time 699761497345 state 0x1 ref 1 00000100:00100000:3.0:1789760718.489223:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699761757761 time 699804526310 state 0x0 ref 1 00000100:00100000:3.0:1789760718.489241:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507180928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.489308:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17686528, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.489320:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.489340:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760718.489345:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.489353:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.489370:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.519620:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971638 is committed 00002000:00010000:2.0:1789760718.519779:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17686528 -> 17690624 00010000:00010000:2.0:1789760718.519799:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519814:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519825:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519832:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.519838:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519849:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519870:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.519879:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b724d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17686528->17690623] (req 17686528->17690623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.520012:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507180928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30758us (31059us total) trans 4294971638 rc 0/0 00000100:00100000:2.0:1789760718.520024:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4348 00000100:00100000:2.0:1789760718.520035:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699761757761 time 699804526310 state 0x2 ref 0 00000100:00100000:2.0:1789760718.523936:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760718.524133:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181056 00000100:00100000:0.0:1789760718.524146:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181056 00000100:00100000:0.0:1789760718.524217:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699761757761 time 699804526310 state 0x0 ref 1 00000100:00100000:0.0:1789760718.524237:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699804786726 time 699804526310 state 0x1 ref 1 00000100:00100000:0.0:1789760718.524320:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699804786726 time 699839622362 state 0x0 ref 1 00000100:00100000:0.0:1789760718.524336:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507181056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760718.525259:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17690624, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:0.0:1789760718.525275:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.525294:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760718.525300:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.525309:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.525326:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.567105:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971639 is committed 00002000:00010000:3.0:1789760718.567288:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17690624 -> 17694720 00010000:00010000:3.0:1789760718.567307:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567327:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567338:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567348:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.567354:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567366:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567390:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.567398:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7254 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.567577:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507181056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43237us (43649us total) trans 4294971639 rc 0/0 00000100:00100000:3.0:1789760718.567592:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4349 00000100:00100000:3.0:1789760718.567605:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699804786726 time 699839622362 state 0x2 ref 0 00000100:00100000:1.0:1789760718.574817:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.574926:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181184 00000100:00100000:3.0:1789760718.574939:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181184 00000100:00100000:3.0:1789760718.575030:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699804786726 time 699839622362 state 0x0 ref 1 00000100:00100000:3.0:1789760718.575049:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699839882778 time 699839622362 state 0x1 ref 1 00000100:00100000:3.0:1789760718.575086:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699839882778 time 699890388678 state 0x0 ref 1 00000100:00100000:3.0:1789760718.575101:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.575171:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17694720, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.575184:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.575205:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760718.575211:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.575219:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.575247:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.634751:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971640 is committed 00002000:00010000:0.0:1789760718.635111:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17694720 -> 17698816 00010000:00010000:0.0:1789760718.635132:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635147:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635158:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635166:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760718.635172:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635184:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635208:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635217:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b725b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.635453:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60348us (60645us total) trans 4294971640 rc 0/0 00000100:00100000:0.0:1789760718.635466:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4350 00000100:00100000:0.0:1789760718.635478:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699839882778 time 699890388678 state 0x2 ref 0 00000100:00100000:0.0:1789760718.642427:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.642508:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181312 00000100:00100000:2.0:1789760718.642521:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181312 00000100:00100000:2.0:1789760718.642599:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699839882778 time 699890388678 state 0x0 ref 1 00000100:00100000:2.0:1789760718.642619:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699890649094 time 699890388678 state 0x1 ref 1 00000100:00100000:2.0:1789760718.642661:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699890649094 time 699957963527 state 0x0 ref 1 00000100:00100000:2.0:1789760718.642677:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507181312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.642766:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17698816, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.642775:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.642790:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760718.642795:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.642801:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.642814:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.727081:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17698816 -> 17702912 00010000:00010000:2.0:1789760718.727106:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727120:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727130:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727138:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.727143:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727152:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727179:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.727187:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7262 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17698816->17702911] (req 17698816->17702911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.727352:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507181312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 84670us (84942us total) trans 4294971641 rc 0/0 00000100:00100000:2.0:1789760718.727365:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4351 00000100:00100000:2.0:1789760718.727377:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699890649094 time 699957963527 state 0x2 ref 0 00000001:00080000:2.0:1789760718.727427:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971641 is committed 00000100:00100000:0.0:1789760718.733860:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.741216:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181440 00000100:00100000:3.0:1789760718.741239:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181440 00000100:00100000:3.0:1789760718.741333:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699890649094 time 699957963527 state 0x0 ref 1 00000100:00100000:3.0:1789760718.741353:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699958223943 time 699957963527 state 0x1 ref 1 00000100:00100000:3.0:1789760718.741396:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699958223943 time 700056699140 state 0x0 ref 1 00000100:00100000:3.0:1789760718.741412:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.741481:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17702912, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.741492:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.741512:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760718.741518:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.741525:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.741548:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.751884:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760718.813789:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17702912 -> 17707008 00010000:00010000:3.0:1789760718.813818:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813834:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813844:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813852:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.813857:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813867:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813894:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.813902:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7269 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.814127:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507181440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72708us (80273us total) trans 4294971642 rc 0/0 00000100:00100000:3.0:1789760718.814142:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4352 00000100:00100000:3.0:1789760718.814155:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699958223943 time 700056699140 state 0x2 ref 0 00000001:00080000:2.0:1789760718.814261:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971642 is committed 00000100:00100000:0.0:1789760718.822254:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.822838:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181568 00000100:00100000:3.0:1789760718.822856:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181568 00000100:00100000:3.0:1789760718.822942:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 699958223943 time 700056699140 state 0x0 ref 1 00000100:00100000:3.0:1789760718.822972:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700056959556 time 700056699140 state 0x1 ref 1 00000100:00100000:3.0:1789760718.823121:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700056959556 time 700138422422 state 0x0 ref 1 00000100:00100000:3.0:1789760718.823148:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.823215:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17707008, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.823226:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.823247:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760718.823253:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.823261:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.823279:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.866689:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17707008 -> 17711104 00010000:00010000:2.0:1789760718.866745:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866759:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866770:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866778:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760718.866783:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866793:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866819:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.866829:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7270 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.866980:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507181568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43829us (44735us total) trans 4294971643 rc 0/0 00000100:00100000:2.0:1789760718.868201:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4353 00000100:00100000:2.0:1789760718.868220:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700056959556 time 700138422422 state 0x2 ref 0 00000001:00080000:2.0:1789760718.868290:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971643 is committed 00000100:00100000:0.0:1789760718.871123:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760718.871717:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181696 00000100:00100000:2.0:1789760718.871730:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181696 00000100:00100000:2.0:1789760718.871808:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700056959556 time 700138422422 state 0x0 ref 1 00000100:00100000:2.0:1789760718.871830:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700138682838 time 700138422422 state 0x1 ref 1 00000100:00100000:2.0:1789760718.871862:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700138682838 time 700187165590 state 0x0 ref 1 00000100:00100000:2.0:1789760718.871879:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760718.871969:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17711104, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:2.0:1789760718.871981:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.872010:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760718.872016:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.872025:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.872043:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.929914:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971644 is committed 00002000:00010000:3.0:1789760718.930196:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17711104 -> 17715200 00010000:00010000:3.0:1789760718.930217:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930232:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930242:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930250:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.930255:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930266:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930292:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.930300:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7277 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17711104->17715199] (req 17711104->17715199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.930511:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507181696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58629us (59398us total) trans 4294971644 rc 0/0 00000100:00100000:3.0:1789760718.930528:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4354 00000100:00100000:3.0:1789760718.930540:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700138682838 time 700187165590 state 0x2 ref 0 00000100:00100000:1.0:1789760718.933759:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.933844:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181824 00000100:00100000:3.0:1789760718.933856:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181824 00000100:00100000:3.0:1789760718.934112:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700138682838 time 700187165590 state 0x0 ref 1 00000100:00100000:3.0:1789760718.934134:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700187426006 time 700187165590 state 0x1 ref 1 00000100:00100000:3.0:1789760718.934210:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700187426006 time 700249511433 state 0x0 ref 1 00000100:00100000:3.0:1789760718.934225:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507181824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.934291:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17715200, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.934302:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.934322:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760718.934327:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.934335:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.934351:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.978723:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971645 is committed 00002000:00010000:3.0:1789760718.978849:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17715200 -> 17719296 00010000:00010000:3.0:1789760718.978869:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978886:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978897:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978906:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760718.978912:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978921:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978966:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.978975:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b727e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.979214:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507181824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44983us (45519us total) trans 4294971645 rc 0/0 00000100:00100000:3.0:1789760718.979228:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4355 00000100:00100000:3.0:1789760718.979240:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700187426006 time 700249511433 state 0x2 ref 0 00000100:00100000:0.0:1789760718.982524:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760718.982687:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507181952 00000100:00100000:3.0:1789760718.982730:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507181952 00000100:00100000:3.0:1789760718.982806:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700187426006 time 700249511433 state 0x0 ref 1 00000100:00100000:3.0:1789760718.982826:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700249771849 time 700249511433 state 0x1 ref 1 00000100:00100000:3.0:1789760718.982852:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700249771849 time 700298155438 state 0x0 ref 1 00000100:00100000:3.0:1789760718.982866:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507181952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760718.982933:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17719296, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760718.982962:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.982980:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760718.982985:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.983002:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.983021:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.025414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971646 is committed 00002000:00010000:3.0:1789760719.025548:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17719296 -> 17723392 00010000:00010000:3.0:1789760719.025569:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025583:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025601:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025609:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.025614:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025624:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025648:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.025657:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7285 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17719296->17723391] (req 17719296->17723391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.026257:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507181952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43379us (43735us total) trans 4294971646 rc 0/0 00000100:00100000:3.0:1789760719.026272:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4356 00000100:00100000:3.0:1789760719.026287:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700249771849 time 700298155438 state 0x2 ref 0 00000100:00100000:0.0:1789760719.030436:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.030630:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182080 00000100:00100000:3.0:1789760719.030643:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182080 00000100:00100000:3.0:1789760719.030793:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700249771849 time 700298155438 state 0x0 ref 1 00000100:00100000:3.0:1789760719.030811:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700298415854 time 700298155438 state 0x1 ref 1 00000100:00100000:3.0:1789760719.030836:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700298415854 time 700346139730 state 0x0 ref 1 00000100:00100000:3.0:1789760719.030849:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.030910:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17723392, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x8807 00010000:00010000:3.0:1789760719.030920:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.030939:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.030961:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.030969:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.030986:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760719.031518:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760719.068853:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760719.068885:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538263936:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760719.069011:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760719.069100:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538263936 00000100:00100000:3.0:1789760719.069109:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538263936 00000100:00100000:3.0:1789760719.069168:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789760719.069180:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538263936:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760719.069210:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3840933888 avail 3833593856 00000100:00100000:3.0:1789760719.069262:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538263936:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (251us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760719.069273:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789760719.069352:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538263936:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760719.069891:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538264064:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760719.069939:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760719.070295:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264064 00000100:00100000:1.0:1789760719.070302:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264064 00000100:00100000:1.0:1789760719.070356:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:1.0:1789760719.070366:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538264064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760719.070387:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3840933888 avail 3833593856 00000100:00100000:1.0:1789760719.070634:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538264064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 268us (692us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760719.070642:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789760719.070746:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538264064:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789760719.075183:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17723392 -> 17727488 00002000:00010000:0.0:1789760719.075196:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:0.0:1789760719.075199:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:0.0:1789760719.075217:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075231:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075241:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075249:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760719.075254:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075274:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075298:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.075306:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b728c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17723392->17727487] (req 17723392->17727487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.075777:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507182080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44921us (45346us total) trans 4294971647 rc 0/0 00000100:00100000:0.0:1789760719.075791:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4357 00000100:00100000:0.0:1789760719.075804:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700298415854 time 700346139730 state 0x2 ref 0 00000001:00080000:1.0:1789760719.076564:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971647 is committed 00000100:00100000:0.0:1789760719.093585:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.094062:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182208 00000100:00100000:3.0:1789760719.094076:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182208 00000100:00100000:3.0:1789760719.094161:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700298415854 time 700346139730 state 0x0 ref 1 00000100:00100000:3.0:1789760719.094180:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700346400146 time 700346139730 state 0x1 ref 1 00000100:00100000:3.0:1789760719.094218:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700346400146 time 700409521056 state 0x0 ref 1 00000100:00100000:3.0:1789760719.094232:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21face850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.094297:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17727488, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.094307:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.094326:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.094332:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.094340:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.094357:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.146140:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971648 is committed 00002000:00010000:3.0:1789760719.146868:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17727488 -> 17731584 00010000:00010000:3.0:1789760719.146890:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146907:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146918:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146926:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.146932:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146943:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146978:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.146987:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7293 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.147259:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21face850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53022us (53681us total) trans 4294971648 rc 0/0 00000100:00100000:3.0:1789760719.147272:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4358 00000100:00100000:3.0:1789760719.147285:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700346400146 time 700409521056 state 0x2 ref 0 00000100:00100000:2.0:1789760719.157373:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.157544:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182336 00000100:00100000:3.0:1789760719.157559:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182336 00000100:00100000:3.0:1789760719.157641:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700346400146 time 700409521056 state 0x0 ref 1 00000100:00100000:3.0:1789760719.157663:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700409781472 time 700409521056 state 0x1 ref 1 00000100:00100000:3.0:1789760719.157728:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700409781472 time 700473029976 state 0x0 ref 1 00000100:00100000:3.0:1789760719.157743:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.157812:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17731584, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.157824:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.157845:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.157850:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.157858:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.157877:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.190682:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971649 is committed 00002000:00010000:3.0:1789760719.190826:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17731584 -> 17735680 00010000:00010000:3.0:1789760719.190846:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190861:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190877:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190886:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.190891:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190900:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190924:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.190931:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b729a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17731584->17735679] (req 17731584->17735679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.191065:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33319us (33712us total) trans 4294971649 rc 0/0 00000100:00100000:3.0:1789760719.191077:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4359 00000100:00100000:3.0:1789760719.191089:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700409781472 time 700473029976 state 0x2 ref 0 00000100:00100000:0.0:1789760719.195031:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.197739:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182464 00000100:00100000:3.0:1789760719.197761:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182464 00000100:00100000:3.0:1789760719.197860:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700409781472 time 700473029976 state 0x0 ref 1 00000100:00100000:3.0:1789760719.197881:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700473290392 time 700473029976 state 0x1 ref 1 00000100:00100000:3.0:1789760719.197910:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700473290392 time 700513213848 state 0x0 ref 1 00000100:00100000:3.0:1789760719.197927:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.198020:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17735680, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.198031:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.198052:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.198057:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.198064:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.198082:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.237652:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971650 is committed 00002000:00010000:3.0:1789760719.237785:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17735680 -> 17739776 00010000:00010000:3.0:1789760719.237805:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237819:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237828:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237836:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.237842:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237851:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237875:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.237882:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17735680->17739775] (req 17735680->17739775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.238031:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40101us (43008us total) trans 4294971650 rc 0/0 00000100:00100000:3.0:1789760719.238045:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4360 00000100:00100000:3.0:1789760719.238058:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700473290392 time 700513213848 state 0x2 ref 0 00000100:00100000:2.0:1789760719.243236:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.243329:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182592 00000100:00100000:3.0:1789760719.243340:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182592 00000100:00100000:3.0:1789760719.243429:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700473290392 time 700513213848 state 0x0 ref 1 00000100:00100000:3.0:1789760719.243449:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700513474264 time 700513213848 state 0x1 ref 1 00000100:00100000:3.0:1789760719.243485:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700513474264 time 700558787546 state 0x0 ref 1 00000100:00100000:3.0:1789760719.243498:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.243564:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17739776, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.243576:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.243597:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.243602:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.243609:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.243638:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.310443:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971651 is committed 00002000:00010000:2.0:1789760719.310642:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17739776 -> 17743872 00010000:00010000:2.0:1789760719.310663:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310679:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310689:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310723:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760719.310730:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310741:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310767:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.310777:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b72a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.310903:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67399us (67676us total) trans 4294971651 rc 0/0 00000100:00100000:2.0:1789760719.310915:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4361 00000100:00100000:2.0:1789760719.310928:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700513474264 time 700558787546 state 0x2 ref 0 00000100:00100000:2.0:1789760719.315566:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.315986:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182720 00000100:00100000:3.0:1789760719.316073:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182720 00000100:00100000:3.0:1789760719.316156:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700513474264 time 700558787546 state 0x0 ref 1 00000100:00100000:3.0:1789760719.316178:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700559047962 time 700558787546 state 0x1 ref 1 00000100:00100000:3.0:1789760719.316214:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700559047962 time 700631516672 state 0x0 ref 1 00000100:00100000:3.0:1789760719.316229:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.316295:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17743872, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.316307:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.316327:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.316333:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.316342:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.316371:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760719.367158:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17743872 -> 17747968 00010000:00010000:2.0:1789760719.367184:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367201:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367212:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367220:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760719.367226:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367237:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367269:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.367278:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b72af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.367501:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507182720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51265us (51941us total) trans 4294971652 rc 0/0 00000100:00100000:2.0:1789760719.367513:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4362 00000100:00100000:2.0:1789760719.367527:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700559047962 time 700631516672 state 0x2 ref 0 00000001:00080000:1.0:1789760719.368850:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971652 is committed 00000100:00100000:3.0:1789760719.371507:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760719.371751:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182848 00000100:00100000:0.0:1789760719.371766:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182848 00000100:00100000:0.0:1789760719.371835:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700559047962 time 700631516672 state 0x0 ref 1 00000100:00100000:0.0:1789760719.371854:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700631777088 time 700631516672 state 0x1 ref 1 00000100:00100000:0.0:1789760719.371891:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700631777088 time 700687194037 state 0x0 ref 1 00000100:00100000:0.0:1789760719.371904:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507182848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760719.371982:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17747968, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:0.0:1789760719.372087:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.372108:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760719.372114:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.372123:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.372142:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.420108:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971653 is committed 00002000:00010000:3.0:1789760719.420116:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17747968 -> 17752064 00010000:00010000:3.0:1789760719.420150:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420166:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420177:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420186:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.420193:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420203:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420227:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.420242:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b72b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.420424:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507182848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48512us (48923us total) trans 4294971653 rc 0/0 00000100:00100000:3.0:1789760719.420445:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4363 00000100:00100000:3.0:1789760719.420458:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700631777088 time 700687194037 state 0x2 ref 0 00000100:00100000:3.0:1789760719.426056:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760719.426137:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507182976 00000100:00100000:2.0:1789760719.426150:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507182976 00000100:00100000:2.0:1789760719.426221:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700631777088 time 700687194037 state 0x0 ref 1 00000100:00100000:2.0:1789760719.426241:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700687454453 time 700687194037 state 0x1 ref 1 00000100:00100000:2.0:1789760719.426280:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700687454453 time 700741582875 state 0x0 ref 1 00000100:00100000:2.0:1789760719.426295:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507182976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760719.426361:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17752064, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:2.0:1789760719.426372:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.426398:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760719.426403:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.426411:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.426427:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.499167:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17752064 -> 17756160 00010000:00010000:0.0:1789760719.499194:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499209:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499219:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499227:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760719.499232:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499242:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499267:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.499274:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b72bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17752064->17756159] (req 17752064->17756159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.499471:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507182976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73171us (73423us total) trans 4294971654 rc 0/0 00000100:00100000:0.0:1789760719.499488:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4364 00000100:00100000:0.0:1789760719.499500:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700687454453 time 700741582875 state 0x2 ref 0 00000001:00080000:3.0:1789760719.501815:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971654 is committed 00000100:00100000:2.0:1789760719.506901:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.507062:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183104 00000100:00100000:3.0:1789760719.507080:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183104 00000100:00100000:3.0:1789760719.507162:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700687454453 time 700741582875 state 0x0 ref 1 00000100:00100000:3.0:1789760719.507181:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700741843291 time 700741582875 state 0x1 ref 1 00000100:00100000:3.0:1789760719.507221:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700741843291 time 700822523824 state 0x0 ref 1 00000100:00100000:3.0:1789760719.507236:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507183104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.507302:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17756160, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.507312:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.507332:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.507337:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.507346:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.507362:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.593140:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760719.593167:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538264192:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760719.593233:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760719.593784:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264192 00000100:00100000:2.0:1789760719.593792:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264192 00000100:00100000:2.0:1789760719.593849:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760719.593859:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538264192:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760719.593897:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760719.593959:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538264192:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (717us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760719.593967:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789760719.594111:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538264192:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760719.609785:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971655 is committed 00002000:00010000:1.0:1789760719.609938:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17756160 -> 17760256 00010000:00010000:1.0:1789760719.609964:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.609977:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.609985:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.610002:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.610007:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.610016:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.610035:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.610042:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b72c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.610143:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507183104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 102905us (103247us total) trans 4294971655 rc 0/0 00000100:00100000:1.0:1789760719.610153:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4365 00000100:00100000:1.0:1789760719.610164:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700741843291 time 700822523824 state 0x2 ref 0 00000100:00100000:1.0:1789760719.618235:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.618353:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183232 00000100:00100000:3.0:1789760719.618363:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183232 00000100:00100000:3.0:1789760719.618419:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700741843291 time 700822523824 state 0x0 ref 1 00000100:00100000:3.0:1789760719.618434:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700822784240 time 700822523824 state 0x1 ref 1 00000100:00100000:3.0:1789760719.618456:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700822784240 time 700933760072 state 0x0 ref 1 00000100:00100000:3.0:1789760719.618469:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507183232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.618517:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17760256, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.618526:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.618541:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760719.618546:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.618552:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.618566:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760719.677511:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17760256 -> 17764352 00010000:00010000:1.0:1789760719.677532:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677544:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677553:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677562:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.677567:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677582:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677601:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.677609:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b72cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17760256->17764351] (req 17760256->17764351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.677736:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507183232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59264us (59507us total) trans 4294971656 rc 0/0 00000100:00100000:1.0:1789760719.677747:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4366 00000100:00100000:1.0:1789760719.677756:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700822784240 time 700933760072 state 0x2 ref 0 00000001:00080000:0.0:1789760719.677970:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971656 is committed 00000100:00100000:0.0:1789760719.680570:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760719.681964:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183360 00000100:00100000:2.0:1789760719.681978:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183360 00000100:00100000:2.0:1789760719.682047:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700822784240 time 700933760072 state 0x0 ref 1 00000100:00100000:2.0:1789760719.682063:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700934020488 time 700933760072 state 0x1 ref 1 00000100:00100000:2.0:1789760719.682095:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700934020488 time 700997398705 state 0x0 ref 1 00000100:00100000:2.0:1789760719.682108:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507183360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760719.682157:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17764352, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:2.0:1789760719.682166:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.682181:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760719.682185:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.682191:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.682203:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.771559:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971657 is committed 00010000:00010000:0.0:1789760719.771864:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760719.771878:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17764352 -> 17768448 00010000:00010000:1.0:1789760719.771893:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771906:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771915:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771922:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.771927:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771937:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771967:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.771975:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b72d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17764352->17768447] (req 17764352->17768447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.772067:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507183360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 89957us (91502us total) trans 4294971657 rc 0/0 00000100:00100000:1.0:1789760719.772078:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4367 00000100:00100000:1.0:1789760719.772087:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700934020488 time 700997398705 state 0x2 ref 0 00000100:00100000:2.0:1789760719.780503:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760719.780565:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183488 00000100:00100000:1.0:1789760719.780575:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183488 00000100:00100000:1.0:1789760719.780632:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700934020488 time 700997398705 state 0x0 ref 1 00000100:00100000:1.0:1789760719.780648:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700997659121 time 700997398705 state 0x1 ref 1 00000100:00100000:1.0:1789760719.780772:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700997659121 time 701096073558 state 0x0 ref 1 00000100:00100000:1.0:1789760719.780789:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507183488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760719.780839:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17768448, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:1.0:1789760719.780848:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.780863:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760719.780867:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.780873:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.780897:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.863115:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971658 is committed 00002000:00010000:1.0:1789760719.863228:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17768448 -> 17772544 00010000:00010000:1.0:1789760719.863240:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863248:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863254:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863258:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.863262:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863268:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863286:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.863291:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b72d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.863370:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507183488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82582us (82874us total) trans 4294971658 rc 0/0 00000100:00100000:1.0:1789760719.863380:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4368 00000100:00100000:1.0:1789760719.863389:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700997659121 time 701096073558 state 0x2 ref 0 00000100:00100000:3.0:1789760719.866132:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760719.866222:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183616 00000100:00100000:1.0:1789760719.866231:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183616 00000100:00100000:1.0:1789760719.866282:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 700997659121 time 701096073558 state 0x0 ref 1 00000100:00100000:1.0:1789760719.866297:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701096333974 time 701096073558 state 0x1 ref 1 00000100:00100000:1.0:1789760719.866329:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701096333974 time 701181632861 state 0x0 ref 1 00000100:00100000:1.0:1789760719.866341:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507183616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760719.866385:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17772544, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:1.0:1789760719.866394:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.866409:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760719.866414:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.866421:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.866434:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.891213:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971659 is committed 00002000:00010000:1.0:1789760719.891249:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17772544 -> 17776640 00010000:00010000:1.0:1789760719.891264:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891277:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891285:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891290:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.891295:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891304:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891322:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.891335:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.891422:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507183616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25078us (25295us total) trans 4294971659 rc 0/0 00000100:00100000:1.0:1789760719.891432:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4369 00000100:00100000:1.0:1789760719.891442:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701096333974 time 701181632861 state 0x2 ref 0 00000100:00100000:0.0:1789760719.898096:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760719.898545:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183744 00000100:00100000:1.0:1789760719.898558:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183744 00000100:00100000:1.0:1789760719.898614:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701096333974 time 701181632861 state 0x0 ref 1 00000100:00100000:1.0:1789760719.898631:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701181893277 time 701181632861 state 0x1 ref 1 00000100:00100000:1.0:1789760719.898660:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701181893277 time 701213963034 state 0x0 ref 1 00000100:00100000:1.0:1789760719.898674:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507183744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760719.898757:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17776640, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:1.0:1789760719.898766:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.898783:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760719.898788:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.898795:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.898816:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.926969:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971660 is committed 00002000:00010000:1.0:1789760719.927148:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17776640 -> 17780736 00010000:00010000:1.0:1789760719.927164:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927177:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927186:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927193:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760719.927198:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927209:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927230:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.927238:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17776640->17780735] (req 17776640->17780735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.927932:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507183744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29255us (29839us total) trans 4294971660 rc 0/0 00000100:00100000:1.0:1789760719.927945:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4370 00000100:00100000:1.0:1789760719.927965:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701181893277 time 701213963034 state 0x2 ref 0 00000100:00100000:2.0:1789760719.933227:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760719.933480:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507183872 00000100:00100000:1.0:1789760719.933490:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507183872 00000100:00100000:1.0:1789760719.933547:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701181893277 time 701213963034 state 0x0 ref 1 00000100:00100000:1.0:1789760719.933563:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701214223450 time 701213963034 state 0x1 ref 1 00000100:00100000:0.0:1789760719.933792:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701214223450 time 701249091932 state 0x0 ref 1 00000100:00100000:0.0:1789760719.933811:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507183872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760719.933861:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17780736, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:0.0:1789760719.933870:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.933887:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760719.933892:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.933899:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.933919:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.981239:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971661 is committed 00002000:00010000:3.0:1789760719.981339:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17780736 -> 17784832 00010000:00010000:3.0:1789760719.981354:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981366:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981375:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981383:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760719.981388:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981461:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981490:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.981498:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b72ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.981606:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507183872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47796us (48384us total) trans 4294971661 rc 0/0 00000100:00100000:3.0:1789760719.981615:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4371 00000100:00100000:3.0:1789760719.981625:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701214223450 time 701249091932 state 0x2 ref 0 00000100:00100000:2.0:1789760719.985328:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760719.985564:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184000 00000100:00100000:3.0:1789760719.985574:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184000 00000100:00100000:3.0:1789760719.985624:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701214223450 time 701249091932 state 0x0 ref 1 00000100:00100000:3.0:1789760719.985639:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701249352348 time 701249091932 state 0x1 ref 1 00000100:00100000:3.0:1789760719.985669:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701249352348 time 701300971751 state 0x0 ref 1 00000100:00100000:3.0:1789760719.985690:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760719.985779:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17784832, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:3.0:1789760719.985788:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.985804:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760719.985809:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.985815:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.985828:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.029527:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971662 is committed 00002000:00010000:1.0:1789760720.029618:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17784832 -> 17788928 00010000:00010000:1.0:1789760720.029635:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029647:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029655:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029662:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760720.029667:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029685:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029726:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.029733:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b72f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.029830:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44137us (44506us total) trans 4294971662 rc 0/0 00000100:00100000:1.0:1789760720.029839:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4372 00000100:00100000:1.0:1789760720.029848:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701249352348 time 701300971751 state 0x2 ref 0 00000100:00100000:0.0:1789760720.033918:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.034132:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184128 00000100:00100000:1.0:1789760720.034142:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184128 00000100:00100000:1.0:1789760720.034196:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701249352348 time 701300971751 state 0x0 ref 1 00000100:00100000:1.0:1789760720.034212:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701301232167 time 701300971751 state 0x1 ref 1 00000100:00100000:1.0:1789760720.034242:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701301232167 time 701349545663 state 0x0 ref 1 00000100:00100000:1.0:1789760720.034254:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507184128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760720.034302:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17788928, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x8807 00010000:00010000:1.0:1789760720.034311:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.034327:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760720.034332:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.034338:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.034352:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.076933:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17788928 -> 17793024 00002000:00010000:1.0:1789760720.076943:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:1.0:1789760720.076957:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:1.0:1789760720.076970:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.076992:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.077002:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.077010:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760720.077015:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.077025:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.077046:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.077055:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b72fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17788928->17793023] (req 17788928->17793023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.077146:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507184128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42888us (43232us total) trans 4294971663 rc 0/0 00000100:00100000:1.0:1789760720.077156:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4373 00000100:00100000:1.0:1789760720.077166:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701301232167 time 701349545663 state 0x2 ref 0 00000001:00080000:0.0:1789760720.077292:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971663 is committed 00000100:00100000:3.0:1789760720.080733:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760720.080824:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184256 00000100:00100000:0.0:1789760720.080834:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184256 00000100:00100000:0.0:1789760720.080886:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701301232167 time 701349545663 state 0x0 ref 1 00000100:00100000:0.0:1789760720.080900:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701349806079 time 701349545663 state 0x1 ref 1 00000100:00100000:0.0:1789760720.080935:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701349806079 time 701396237270 state 0x0 ref 1 00000100:00100000:0.0:1789760720.080957:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760720.081015:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17793024, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:0.0:1789760720.081024:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.081039:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760720.081043:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.081050:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.081063:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.092768:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538264320:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760720.092841:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760720.092938:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264320 00000100:00100000:3.0:1789760720.092946:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264320 00000100:00100000:3.0:1789760720.093011:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:3.0:1789760720.093020:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538264320:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760720.093054:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760720.093100:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538264320:12345-0@lo:41:osp-pre-0-1.0 Request processed in 81us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760720.093107:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:3.0:1789760720.093156:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538264320:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760720.114786:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971664 is committed 00002000:00010000:2.0:1789760720.114960:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17793024 -> 17797120 00010000:00010000:2.0:1789760720.114976:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.114999:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.115007:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.115019:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760720.115023:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.115032:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.115051:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.115058:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7303 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.115150:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34199us (34420us total) trans 4294971664 rc 0/0 00000100:00100000:2.0:1789760720.115161:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4374 00000100:00100000:2.0:1789760720.115169:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701349806079 time 701396237270 state 0x2 ref 0 00000100:00100000:2.0:1789760720.119639:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.119770:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184384 00000100:00100000:3.0:1789760720.119780:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184384 00000100:00100000:3.0:1789760720.119834:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701349806079 time 701396237270 state 0x0 ref 1 00000100:00100000:3.0:1789760720.119850:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701396497686 time 701396237270 state 0x1 ref 1 00000100:00100000:3.0:1789760720.119879:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701396497686 time 701435181836 state 0x0 ref 1 00000100:00100000:3.0:1789760720.119891:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507184384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.119933:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17797120, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:3.0:1789760720.119941:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.119972:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760720.119976:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.119990:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.120005:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.158286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971665 is committed 00002000:00010000:3.0:1789760720.158385:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17797120 -> 17801216 00010000:00010000:3.0:1789760720.158401:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158414:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158423:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158430:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.158434:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158443:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158462:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.158471:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b730a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17797120->17801215] (req 17797120->17801215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.158560:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507184384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38667us (38926us total) trans 4294971665 rc 0/0 00000100:00100000:3.0:1789760720.158570:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4375 00000100:00100000:3.0:1789760720.158580:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701396497686 time 701435181836 state 0x2 ref 0 00000100:00100000:3.0:1789760720.162379:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760720.162582:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184512 00000100:00100000:2.0:1789760720.162592:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184512 00000100:00100000:2.0:1789760720.162647:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701396497686 time 701435181836 state 0x0 ref 1 00000100:00100000:2.0:1789760720.162662:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701435442252 time 701435181836 state 0x1 ref 1 00000100:00100000:2.0:1789760720.162830:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701435442252 time 701478132075 state 0x0 ref 1 00000100:00100000:2.0:1789760720.162846:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760720.162896:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17801216, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:2.0:1789760720.162905:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.162921:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760720.162925:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.162933:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.162959:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.206084:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971666 is committed 00002000:00010000:2.0:1789760720.206155:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17801216 -> 17805312 00010000:00010000:2.0:1789760720.206170:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206184:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206194:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206202:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760720.206207:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206218:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206237:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.206245:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7311 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17801216->17805311] (req 17801216->17805311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.206918:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44068us (44541us total) trans 4294971666 rc 0/0 00000100:00100000:2.0:1789760720.206940:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4376 00000100:00100000:2.0:1789760720.206959:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701435442252 time 701478132075 state 0x2 ref 0 00000100:00100000:2.0:1789760720.219124:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760720.219220:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184640 00000100:00100000:0.0:1789760720.219231:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184640 00000100:00100000:0.0:1789760720.219285:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701435442252 time 701478132075 state 0x0 ref 1 00000100:00100000:0.0:1789760720.219498:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701478392491 time 701478132075 state 0x1 ref 1 00000100:00100000:0.0:1789760720.219560:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701478392491 time 701534862188 state 0x0 ref 1 00000100:00100000:0.0:1789760720.219573:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760720.219624:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17805312, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:0.0:1789760720.219632:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.219647:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760720.219652:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.219660:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.219839:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.272488:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971667 is committed 00002000:00010000:3.0:1789760720.272626:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17805312 -> 17809408 00010000:00010000:3.0:1789760720.272640:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272651:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272660:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272666:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.272682:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272710:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272735:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.272743:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7318 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.272976:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53399us (53856us total) trans 4294971667 rc 0/0 00000100:00100000:3.0:1789760720.272995:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4377 00000100:00100000:3.0:1789760720.273005:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701478392491 time 701534862188 state 0x2 ref 0 00000100:00100000:2.0:1789760720.277567:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.277652:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184768 00000100:00100000:3.0:1789760720.277661:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184768 00000100:00100000:3.0:1789760720.281335:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701478392491 time 701534862188 state 0x0 ref 1 00000100:00100000:3.0:1789760720.281355:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701535122604 time 701534862188 state 0x1 ref 1 00000100:00100000:3.0:1789760720.281386:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701535122604 time 701596689439 state 0x0 ref 1 00000100:00100000:3.0:1789760720.281514:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507184768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.281573:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17809408, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:3.0:1789760720.281583:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.281599:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760720.281604:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.281611:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.281625:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.332867:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17809408 -> 17813504 00010000:00010000:1.0:1789760720.332888:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332901:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332911:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332918:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760720.332923:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332931:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332963:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.332971:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b731f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17809408->17813503] (req 17809408->17813503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.333128:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507184768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51616us (55565us total) trans 4294971668 rc 0/0 00000100:00100000:1.0:1789760720.333139:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4378 00000100:00100000:1.0:1789760720.333148:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701535122604 time 701596689439 state 0x2 ref 0 00000001:00080000:3.0:1789760720.337567:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971668 is committed 00000100:00100000:0.0:1789760720.338506:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.338594:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507184896 00000100:00100000:1.0:1789760720.338604:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507184896 00000100:00100000:1.0:1789760720.338659:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701535122604 time 701596689439 state 0x0 ref 1 00000100:00100000:1.0:1789760720.338675:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701596949855 time 701596689439 state 0x1 ref 1 00000100:00100000:3.0:1789760720.338796:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701596949855 time 701654096803 state 0x0 ref 1 00000100:00100000:3.0:1789760720.338810:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.338855:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17813504, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:3.0:1789760720.338864:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.338879:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760720.338884:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.338894:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.338909:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.391751:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971669 is committed 00002000:00010000:0.0:1789760720.391912:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17813504 -> 17817600 00010000:00010000:0.0:1789760720.391928:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.391942:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.391963:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.391970:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760720.391975:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.391994:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.392014:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.392022:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7326 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17813504->17817599] (req 17813504->17817599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.392112:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507184896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53299us (53715us total) trans 4294971669 rc 0/0 00000100:00100000:0.0:1789760720.392122:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4379 00000100:00100000:0.0:1789760720.392131:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701596949855 time 701654096803 state 0x2 ref 0 00000100:00100000:0.0:1789760720.395755:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.395891:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185024 00000100:00100000:1.0:1789760720.395900:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185024 00000100:00100000:1.0:1789760720.395965:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701596949855 time 701654096803 state 0x0 ref 1 00000100:00100000:1.0:1789760720.395987:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701654357219 time 701654096803 state 0x1 ref 1 00000100:00100000:1.0:1789760720.396012:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701654357219 time 701711315987 state 0x0 ref 1 00000100:00100000:1.0:1789760720.396023:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760720.396067:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17817600, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:1.0:1789760720.396075:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.396090:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760720.396094:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.396100:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.396115:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760720.455319:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17817600 -> 17821696 00010000:00010000:3.0:1789760720.455348:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455367:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455376:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455384:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.455388:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455404:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455424:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.455432:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b732d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.455522:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507185024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59495us (59771us total) trans 4294971670 rc 0/0 00000100:00100000:3.0:1789760720.455532:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4380 00000100:00100000:3.0:1789760720.455541:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701654357219 time 701711315987 state 0x2 ref 0 00000001:00080000:3.0:1789760720.455579:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971670 is committed 00000100:00100000:2.0:1789760720.458631:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.458733:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185152 00000100:00100000:1.0:1789760720.458743:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185152 00000100:00100000:1.0:1789760720.458797:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701654357219 time 701711315987 state 0x0 ref 1 00000100:00100000:1.0:1789760720.458812:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701711576403 time 701711315987 state 0x1 ref 1 00000100:00100000:1.0:1789760720.458844:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701711576403 time 701774146701 state 0x0 ref 1 00000100:00100000:1.0:1789760720.458856:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760720.458903:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17821696, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:1.0:1789760720.458912:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.458927:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760720.458932:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.458938:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.458964:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.516972:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971671 is committed 00002000:00010000:2.0:1789760720.525596:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17821696 -> 17825792 00010000:00010000:2.0:1789760720.525625:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525639:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525650:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525659:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760720.525664:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525674:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525712:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.525721:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7334 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17821696->17825791] (req 17821696->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.525825:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66965us (67196us total) trans 4294971671 rc 0/0 00000100:00100000:2.0:1789760720.525836:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4381 00000100:00100000:2.0:1789760720.525846:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701711576403 time 701774146701 state 0x2 ref 0 00000100:00100000:0.0:1789760720.532470:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.532610:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185280 00000100:00100000:3.0:1789760720.532620:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185280 00000100:00100000:3.0:1789760720.532675:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701711576403 time 701774146701 state 0x0 ref 1 00000100:00100000:3.0:1789760720.532710:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701774407117 time 701774146701 state 0x1 ref 1 00000100:00100000:3.0:1789760720.532738:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701774407117 time 701848042081 state 0x0 ref 1 00000100:00100000:3.0:1789760720.532751:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507185280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.532804:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17825792, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x8807 00010000:00010000:3.0:1789760720.532813:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.532829:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760720.532834:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.532840:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.532855:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.597792:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971672 is committed 00002000:00010000:1.0:1789760720.598108:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17825792 -> 17829888 00002000:00010000:1.0:1789760720.598114:0:8091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:1.0:1789760720.598127:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598140:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598149:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598155:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760720.598160:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598169:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598189:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.598203:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b733b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->17829887] (req 17825792->17829887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.598335:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507185280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65582us (65870us total) trans 4294971672 rc 0/0 00000100:00100000:1.0:1789760720.598346:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4382 00000100:00100000:1.0:1789760720.598356:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701774407117 time 701848042081 state 0x2 ref 0 00000100:00100000:1.0:1789760720.605167:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.605260:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185408 00000100:00100000:3.0:1789760720.605270:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185408 00000100:00100000:3.0:1789760720.605321:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701774407117 time 701848042081 state 0x0 ref 1 00000100:00100000:3.0:1789760720.605337:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701848302497 time 701848042081 state 0x1 ref 1 00000100:00100000:3.0:1789760720.605407:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701848302497 time 701920709272 state 0x0 ref 1 00000100:00100000:3.0:1789760720.605420:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507185408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.605466:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17829888, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:3.0:1789760720.605474:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.605500:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760720.605504:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.605510:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.605524:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760720.642860:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17829888 -> 17833984 00010000:00010000:3.0:1789760720.642878:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642891:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642900:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642906:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.642911:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642919:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642937:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.642957:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7342 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.643049:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507185408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37626us (37886us total) trans 4294971673 rc 0/0 00000100:00100000:3.0:1789760720.643059:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4383 00000100:00100000:3.0:1789760720.643069:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701848302497 time 701920709272 state 0x2 ref 0 00000001:00080000:0.0:1789760720.643133:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971673 is committed 00000100:00100000:2.0:1789760720.650644:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.650761:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185536 00000100:00100000:1.0:1789760720.650771:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185536 00000100:00100000:1.0:1789760720.650825:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701848302497 time 701920709272 state 0x0 ref 1 00000100:00100000:1.0:1789760720.650840:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701920969688 time 701920709272 state 0x1 ref 1 00000100:00100000:1.0:1789760720.650872:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701920969688 time 701966174777 state 0x0 ref 1 00000100:00100000:1.0:1789760720.650885:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760720.650931:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17833984, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:1.0:1789760720.650940:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.650969:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760720.650973:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.650990:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.651006:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760720.716345:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17833984 -> 17838080 00010000:00010000:3.0:1789760720.716365:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716379:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716388:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716397:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.716403:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716412:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716435:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.716444:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7349 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17833984->17838079] (req 17833984->17838079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.716542:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507185536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65654us (65902us total) trans 4294971674 rc 0/0 00000100:00100000:3.0:1789760720.716553:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4384 00000100:00100000:3.0:1789760720.716564:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701920969688 time 701966174777 state 0x2 ref 0 00000001:00080000:2.0:1789760720.716802:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971674 is committed 00000100:00100000:0.0:1789760720.723737:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760720.723819:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185664 00000100:00100000:1.0:1789760720.723829:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185664 00000100:00100000:1.0:1789760720.723884:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701920969688 time 701966174777 state 0x0 ref 1 00000100:00100000:1.0:1789760720.723899:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701966435193 time 701966174777 state 0x1 ref 1 00000100:00100000:1.0:1789760720.723929:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701966435193 time 702039232675 state 0x0 ref 1 00000100:00100000:1.0:1789760720.723940:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507185664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760720.724092:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17838080, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:1.0:1789760720.724102:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.724118:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760720.724122:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.724127:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.724141:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.784971:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971675 is committed 00002000:00010000:1.0:1789760720.785110:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17838080 -> 17842176 00010000:00010000:1.0:1789760720.785125:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785200:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785211:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785219:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760720.785223:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785231:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785250:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.785258:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7350 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17838080->17842175] (req 17838080->17842175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.785497:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507185664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61553us (61804us total) trans 4294971675 rc 0/0 00000100:00100000:1.0:1789760720.785507:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4385 00000100:00100000:1.0:1789760720.785517:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701966435193 time 702039232675 state 0x2 ref 0 00000100:00100000:1.0:1789760720.793599:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.793677:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185792 00000100:00100000:3.0:1789760720.793686:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185792 00000100:00100000:3.0:1789760720.793764:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 701966435193 time 702039232675 state 0x0 ref 1 00000100:00100000:3.0:1789760720.793778:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702039493091 time 702039232675 state 0x1 ref 1 00000100:00100000:3.0:1789760720.793799:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702039493091 time 702109102639 state 0x0 ref 1 00000100:00100000:3.0:1789760720.793810:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.793854:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17842176, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:3.0:1789760720.793863:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.793877:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760720.793881:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.793887:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.793900:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.798753:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760720.883092:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971676 is committed 00002000:00010000:3.0:1789760720.883373:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17842176 -> 17846272 00010000:00010000:3.0:1789760720.883390:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883405:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883416:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883424:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.883429:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883438:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883462:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.883471:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7357 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.883573:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507185792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 89759us (89977us total) trans 4294971676 rc 0/0 00000100:00100000:3.0:1789760720.883584:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4386 00000100:00100000:3.0:1789760720.883593:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702039493091 time 702109102639 state 0x2 ref 0 00000100:00100000:2.0:1789760720.891828:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.891960:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507185920 00000100:00100000:3.0:1789760720.891970:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507185920 00000100:00100000:3.0:1789760720.892268:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702039493091 time 702109102639 state 0x0 ref 1 00000100:00100000:3.0:1789760720.892284:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702109363055 time 702109102639 state 0x1 ref 1 00000100:00100000:3.0:1789760720.892350:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702109363055 time 702207651603 state 0x0 ref 1 00000100:00100000:3.0:1789760720.892363:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507185920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.892413:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17846272, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:3.0:1789760720.892421:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.892437:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760720.892441:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.892447:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.892462:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.972444:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971677 is committed 00002000:00010000:3.0:1789760720.972565:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17846272 -> 17850368 00010000:00010000:3.0:1789760720.972591:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972603:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972612:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972618:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760720.972623:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972631:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972649:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.972660:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b735e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17846272->17850367] (req 17846272->17850367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.972799:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507185920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 80434us (80977us total) trans 4294971677 rc 0/0 00000100:00100000:3.0:1789760720.972809:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4387 00000100:00100000:3.0:1789760720.972818:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702109363055 time 702207651603 state 0x2 ref 0 00000100:00100000:1.0:1789760720.979068:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760720.979153:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186048 00000100:00100000:3.0:1789760720.979165:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186048 00000100:00100000:3.0:1789760720.979218:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702109363055 time 702207651603 state 0x0 ref 1 00000100:00100000:3.0:1789760720.979235:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702207912019 time 702207651603 state 0x1 ref 1 00000100:00100000:3.0:1789760720.979264:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702207912019 time 702294567967 state 0x0 ref 1 00000100:00100000:3.0:1789760720.979277:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507186048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760720.979322:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17850368, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:3.0:1789760720.979331:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.979346:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760720.979351:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.979357:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.979377:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.098169:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971678 is committed 00002000:00010000:3.0:1789760721.099165:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17850368 -> 17854464 00010000:00010000:3.0:1789760721.099184:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099197:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099206:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099213:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.099217:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099226:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099244:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.099251:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7365 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17850368->17854463] (req 17850368->17854463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.099367:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507186048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 120086us (120302us total) trans 4294971678 rc 0/0 00000100:00100000:3.0:1789760721.099377:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4388 00000100:00100000:3.0:1789760721.099387:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702207912019 time 702294567967 state 0x2 ref 0 00000100:00100000:1.0:1789760721.104804:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.104908:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186176 00000100:00100000:3.0:1789760721.104919:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186176 00000100:00100000:3.0:1789760721.104997:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702207912019 time 702294567967 state 0x0 ref 1 00000100:00100000:3.0:1789760721.105012:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702294828383 time 702294567967 state 0x1 ref 1 00000100:00100000:3.0:1789760721.105035:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702294828383 time 702420338550 state 0x0 ref 1 00000100:00100000:3.0:1789760721.105046:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507186176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.105089:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17854464, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9007 00010000:00010000:3.0:1789760721.105098:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.105113:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.105117:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.105124:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.105138:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760721.105403:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760721.162317:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971679 is committed 00002000:00010000:3.0:1789760721.162425:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17854464 -> 17858560 00002000:00010000:3.0:1789760721.162430:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:3.0:1789760721.162433:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:3.0:1789760721.162445:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162459:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162469:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162477:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.162482:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162491:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162511:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.162519:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b736c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17854464->17858559] (req 17854464->17858559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.162623:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507186176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57573us (57825us total) trans 4294971679 rc 0/0 00000100:00100000:3.0:1789760721.162634:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4389 00000100:00100000:3.0:1789760721.162644:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702294828383 time 702420338550 state 0x2 ref 0 00000100:00100000:2.0:1789760721.166581:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.166650:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186304 00000100:00100000:3.0:1789760721.166660:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186304 00000100:00100000:3.0:1789760721.166740:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702294828383 time 702420338550 state 0x0 ref 1 00000100:00100000:3.0:1789760721.166753:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702420598966 time 702420338550 state 0x1 ref 1 00000100:00100000:3.0:1789760721.166772:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702420598966 time 702482076588 state 0x0 ref 1 00000100:00100000:3.0:1789760721.166783:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507186304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.166827:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17858560, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.166835:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.166850:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.166854:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.166860:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.166873:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.214591:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17858560 -> 17862656 00010000:00010000:3.0:1789760721.214610:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214623:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214632:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214639:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.214643:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214652:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214672:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.214680:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7373 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17858560->17862655] (req 17858560->17862655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.214804:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507186304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48017us (48226us total) trans 4294971680 rc 0/0 00000100:00100000:3.0:1789760721.214816:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4390 00000100:00100000:3.0:1789760721.214825:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702420598966 time 702482076588 state 0x2 ref 0 00000001:00080000:2.0:1789760721.214902:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971680 is committed 00000100:00100000:0.0:1789760721.221761:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.221931:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186432 00000100:00100000:3.0:1789760721.221950:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186432 00000100:00100000:3.0:1789760721.222003:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702420598966 time 702482076588 state 0x0 ref 1 00000100:00100000:3.0:1789760721.222019:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702482337004 time 702482076588 state 0x1 ref 1 00000100:00100000:3.0:1789760721.222047:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702482337004 time 702537351279 state 0x0 ref 1 00000100:00100000:3.0:1789760721.222060:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507186432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.222105:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17862656, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.222114:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.222138:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.222150:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.222157:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.222172:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.260553:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17862656 -> 17866752 00010000:00010000:3.0:1789760721.260572:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260586:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260595:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260602:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.260616:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260625:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260644:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.260652:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b737a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.260775:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507186432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38712us (39017us total) trans 4294971681 rc 0/0 00000100:00100000:3.0:1789760721.260784:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4391 00000100:00100000:3.0:1789760721.260794:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702482337004 time 702537351279 state 0x2 ref 0 00000001:00080000:2.0:1789760721.263535:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971681 is committed 00000100:00100000:1.0:1789760721.268834:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.269165:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186560 00000100:00100000:3.0:1789760721.269175:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186560 00000100:00100000:3.0:1789760721.269230:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702482337004 time 702537351279 state 0x0 ref 1 00000100:00100000:3.0:1789760721.269247:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702537611695 time 702537351279 state 0x1 ref 1 00000100:00100000:3.0:1789760721.269280:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702537611695 time 702584583197 state 0x0 ref 1 00000100:00100000:3.0:1789760721.269293:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507186560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.269541:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17866752, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.269550:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.269565:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.269570:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.269577:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.269592:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.321733:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971682 is committed 00002000:00010000:3.0:1789760721.321841:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17866752 -> 17870848 00010000:00010000:3.0:1789760721.321857:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321869:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321882:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321890:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.321894:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321903:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321922:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.321929:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7381 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17866752->17870847] (req 17866752->17870847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.322122:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507186560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52826us (53293us total) trans 4294971682 rc 0/0 00000100:00100000:3.0:1789760721.322132:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4392 00000100:00100000:3.0:1789760721.322142:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702537611695 time 702584583197 state 0x2 ref 0 00000100:00100000:3.0:1789760721.327971:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760721.328152:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186688 00000100:00100000:1.0:1789760721.328161:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186688 00000100:00100000:1.0:1789760721.328214:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702537611695 time 702584583197 state 0x0 ref 1 00000100:00100000:1.0:1789760721.328226:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702584843613 time 702584583197 state 0x1 ref 1 00000100:00100000:1.0:1789760721.328256:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702584843613 time 702643558245 state 0x0 ref 1 00000100:00100000:1.0:1789760721.328268:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507186688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760721.328316:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17870848, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:1.0:1789760721.328325:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.328341:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760721.328346:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.328352:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.328366:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.365520:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971683 is committed 00002000:00010000:3.0:1789760721.365713:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17870848 -> 17874944 00010000:00010000:3.0:1789760721.365730:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365750:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365760:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365767:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.365774:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365782:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365804:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.365811:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7388 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.366134:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507186688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37862us (38166us total) trans 4294971683 rc 0/0 00000100:00100000:3.0:1789760721.366144:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4393 00000100:00100000:3.0:1789760721.366153:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702584843613 time 702643558245 state 0x2 ref 0 00000100:00100000:0.0:1789760721.369181:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.369519:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186816 00000100:00100000:3.0:1789760721.369530:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186816 00000100:00100000:3.0:1789760721.369588:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702584843613 time 702643558245 state 0x0 ref 1 00000100:00100000:3.0:1789760721.369682:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702643818661 time 702643558245 state 0x1 ref 1 00000100:00100000:3.0:1789760721.369788:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702643818661 time 702685091044 state 0x0 ref 1 00000100:00100000:3.0:1789760721.369804:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507186816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.369872:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17874944, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.369881:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.369897:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.369902:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.369909:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.369925:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.394523:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17874944 -> 17879040 00010000:00010000:3.0:1789760721.394543:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394556:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394570:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394577:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.394581:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394589:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394608:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.394615:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b738f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17874944->17879039] (req 17874944->17879039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.394748:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507186816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24943us (25575us total) trans 4294971684 rc 0/0 00000100:00100000:3.0:1789760721.394758:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4394 00000100:00100000:3.0:1789760721.394766:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702643818661 time 702685091044 state 0x2 ref 0 00000001:00080000:0.0:1789760721.396138:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971684 is committed 00000100:00100000:1.0:1789760721.399459:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.399533:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507186944 00000100:00100000:3.0:1789760721.399542:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507186944 00000100:00100000:3.0:1789760721.399595:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702643818661 time 702685091044 state 0x0 ref 1 00000100:00100000:3.0:1789760721.399610:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702685351460 time 702685091044 state 0x1 ref 1 00000100:00100000:3.0:1789760721.399641:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702685351460 time 702714944329 state 0x0 ref 1 00000100:00100000:3.0:1789760721.399653:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507186944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.399740:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17879040, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.399749:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.399764:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.399769:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.399775:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.399789:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.425330:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971685 is committed 00002000:00010000:3.0:1789760721.425523:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17879040 -> 17883136 00010000:00010000:3.0:1789760721.425538:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425551:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425560:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425567:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.425573:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425583:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425603:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.425610:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7396 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.425736:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507186944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26080us (26281us total) trans 4294971685 rc 0/0 00000100:00100000:3.0:1789760721.425746:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4395 00000100:00100000:3.0:1789760721.425756:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702685351460 time 702714944329 state 0x2 ref 0 00000100:00100000:2.0:1789760721.429257:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.429330:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187072 00000100:00100000:3.0:1789760721.429339:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187072 00000100:00100000:3.0:1789760721.429443:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702685351460 time 702714944329 state 0x0 ref 1 00000100:00100000:3.0:1789760721.429459:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702715204745 time 702714944329 state 0x1 ref 1 00000100:00100000:3.0:1789760721.429488:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702715204745 time 702744791204 state 0x0 ref 1 00000100:00100000:3.0:1789760721.429501:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.429549:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17883136, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.429568:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.429584:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.429588:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.429594:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.429607:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.466850:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971686 is committed 00002000:00010000:3.0:1789760721.467180:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17883136 -> 17887232 00010000:00010000:3.0:1789760721.467196:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467210:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467220:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467227:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.467233:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467242:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467266:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.467274:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b739d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.467374:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37871us (38121us total) trans 4294971686 rc 0/0 00000100:00100000:3.0:1789760721.468232:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4396 00000100:00100000:3.0:1789760721.468244:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702715204745 time 702744791204 state 0x2 ref 0 00000100:00100000:2.0:1789760721.473952:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.474313:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187200 00000100:00100000:3.0:1789760721.474325:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187200 00000100:00100000:3.0:1789760721.474378:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702715204745 time 702744791204 state 0x0 ref 1 00000100:00100000:3.0:1789760721.474455:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702745051620 time 702744791204 state 0x1 ref 1 00000100:00100000:3.0:1789760721.474526:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702745051620 time 702789828628 state 0x0 ref 1 00000100:00100000:3.0:1789760721.474602:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.474653:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17887232, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.474662:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.474677:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.474683:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.474689:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.474737:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.531318:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971687 is committed 00002000:00010000:0.0:1789760721.531398:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17887232 -> 17891328 00010000:00010000:0.0:1789760721.531415:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531430:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531440:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531448:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760721.531453:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531463:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531486:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.531493:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b73a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.531734:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57192us (57795us total) trans 4294971687 rc 0/0 00000100:00100000:0.0:1789760721.531745:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4397 00000100:00100000:0.0:1789760721.531755:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702745051620 time 702789828628 state 0x2 ref 0 00000100:00100000:1.0:1789760721.535874:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.535964:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187328 00000100:00100000:3.0:1789760721.535982:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187328 00000100:00100000:3.0:1789760721.536037:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702745051620 time 702789828628 state 0x0 ref 1 00000100:00100000:3.0:1789760721.536052:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702790089044 time 702789828628 state 0x1 ref 1 00000100:00100000:3.0:1789760721.536079:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702790089044 time 702851382257 state 0x0 ref 1 00000100:00100000:3.0:1789760721.536091:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507187328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.536140:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17891328, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.536148:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.536164:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.536168:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.536175:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.536189:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.570952:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971688 is committed 00002000:00010000:1.0:1789760721.571202:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17891328 -> 17895424 00010000:00010000:1.0:1789760721.571222:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571236:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571245:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571251:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760721.571256:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571266:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571286:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.571293:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b73ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17891328->17895423] (req 17891328->17895423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.572405:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507187328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36305us (36531us total) trans 4294971688 rc 0/0 00000100:00100000:1.0:1789760721.572418:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4398 00000100:00100000:1.0:1789760721.572428:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702790089044 time 702851382257 state 0x2 ref 0 00000100:00100000:2.0:1789760721.578743:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.578805:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187456 00000100:00100000:3.0:1789760721.578814:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187456 00000100:00100000:3.0:1789760721.578864:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702790089044 time 702851382257 state 0x0 ref 1 00000100:00100000:3.0:1789760721.578879:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702851642673 time 702851382257 state 0x1 ref 1 00000100:00100000:3.0:1789760721.578913:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702851642673 time 702894215947 state 0x0 ref 1 00000100:00100000:3.0:1789760721.578925:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507187456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.578991:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17895424, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.579001:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.579017:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760721.579021:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.579028:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.579042:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.642376:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971689 is committed 00002000:00010000:2.0:1789760721.642475:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17895424 -> 17899520 00010000:00010000:2.0:1789760721.642489:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642502:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642511:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642519:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760721.642524:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642533:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642623:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.642631:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b73b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17895424->17899519] (req 17895424->17899519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.642846:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507187456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63917us (64116us total) trans 4294971689 rc 0/0 00000100:00100000:2.0:1789760721.642857:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4399 00000100:00100000:2.0:1789760721.642866:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702851642673 time 702894215947 state 0x2 ref 0 00000100:00100000:0.0:1789760721.664773:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.669467:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187584 00000100:00100000:3.0:1789760721.669481:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187584 00000100:00100000:2.0:1789760721.669726:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702851642673 time 702894215947 state 0x0 ref 1 00000100:00100000:2.0:1789760721.669741:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702894476363 time 702894215947 state 0x1 ref 1 00000100:00100000:2.0:1789760721.669774:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702894476363 time 702985076402 state 0x0 ref 1 00000100:00100000:2.0:1789760721.669788:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760721.669837:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17899520, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:2.0:1789760721.669846:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.669864:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760721.669868:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.669875:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.669889:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.749257:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971690 is committed 00002000:00010000:1.0:1789760721.749430:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17899520 -> 17903616 00010000:00010000:1.0:1789760721.749453:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749467:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749475:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749483:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760721.749488:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749496:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749517:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.749531:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b73b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17899520->17903615] (req 17899520->17903615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.749618:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507187584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79829us (84849us total) trans 4294971690 rc 0/0 00000100:00100000:1.0:1789760721.749628:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4400 00000100:00100000:1.0:1789760721.749636:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702894476363 time 702985076402 state 0x2 ref 0 00000100:00100000:0.0:1789760721.752590:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760721.752674:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187712 00000100:00100000:1.0:1789760721.752684:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187712 00000100:00100000:1.0:1789760721.752779:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702894476363 time 702985076402 state 0x0 ref 1 00000100:00100000:1.0:1789760721.752795:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702985336818 time 702985076402 state 0x1 ref 1 00000100:00100000:1.0:1789760721.752815:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702985336818 time 703068118834 state 0x0 ref 1 00000100:00100000:1.0:1789760721.752828:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507187712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760721.752874:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17903616, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:1.0:1789760721.752884:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.752900:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760721.752905:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.752912:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.752932:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.784365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971691 is committed 00002000:00010000:1.0:1789760721.784596:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17903616 -> 17907712 00010000:00010000:1.0:1789760721.784611:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784623:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784632:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784638:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760721.784642:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784651:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784669:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.784676:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.784961:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507187712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32129us (32374us total) trans 4294971691 rc 0/0 00000100:00100000:2.0:1789760721.784986:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4401 00000100:00100000:2.0:1789760721.784995:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702985336818 time 703068118834 state 0x2 ref 0 00000100:00100000:2.0:1789760721.789225:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.789335:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187840 00000100:00100000:3.0:1789760721.789345:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187840 00000100:00100000:3.0:1789760721.789480:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 702985336818 time 703068118834 state 0x0 ref 1 00000100:00100000:3.0:1789760721.789496:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703068379250 time 703068118834 state 0x1 ref 1 00000100:00100000:3.0:1789760721.789565:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703068379250 time 703104867777 state 0x0 ref 1 00000100:00100000:3.0:1789760721.789578:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507187840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760721.789626:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17907712, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:3.0:1789760721.789635:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.789651:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760721.789656:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.789662:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.789676:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.818717:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971692 is committed 00002000:00010000:2.0:1789760721.818820:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17907712 -> 17911808 00010000:00010000:2.0:1789760721.818835:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818848:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818858:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818866:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760721.818871:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818880:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818900:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818908:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b73c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17907712->17911807] (req 17907712->17911807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.819065:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507187840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29483us (29844us total) trans 4294971692 rc 0/0 00000100:00100000:2.0:1789760721.819076:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4402 00000100:00100000:2.0:1789760721.819085:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703068379250 time 703104867777 state 0x2 ref 0 00010000:00010000:3.0:1789760721.821868:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760721.828464:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760721.828603:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507187968 00000100:00100000:1.0:1789760721.828612:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507187968 00000100:00100000:1.0:1789760721.828661:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703068379250 time 703104867777 state 0x0 ref 1 00000100:00100000:1.0:1789760721.828678:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703105128193 time 703104867777 state 0x1 ref 1 00000100:00100000:1.0:1789760721.828744:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703105128193 time 703144046945 state 0x0 ref 1 00000100:00100000:1.0:1789760721.828757:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507187968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760721.828805:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17911808, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:1.0:1789760721.828814:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.828830:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760721.828835:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.828842:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.828856:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.884194:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17911808 -> 17915904 00010000:00010000:3.0:1789760721.884216:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884228:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884238:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884245:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.884250:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884258:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884278:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.884285:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b73ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17911808->17915903] (req 17911808->17915903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.884373:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507187968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55613us (55914us total) trans 4294971693 rc 0/0 00000100:00100000:3.0:1789760721.884457:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4403 00000100:00100000:3.0:1789760721.884467:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703105128193 time 703144046945 state 0x2 ref 0 00000001:00080000:0.0:1789760721.884545:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971693 is committed 00000100:00100000:0.0:1789760721.890096:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760721.890195:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188096 00000100:00100000:2.0:1789760721.890206:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188096 00000100:00100000:2.0:1789760721.890264:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703105128193 time 703144046945 state 0x0 ref 1 00000100:00100000:2.0:1789760721.890279:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703144307361 time 703144046945 state 0x1 ref 1 00000100:00100000:2.0:1789760721.890339:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703144307361 time 703205641604 state 0x0 ref 1 00000100:00100000:2.0:1789760721.890352:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507188096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760721.890411:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17915904, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:2.0:1789760721.890420:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.890435:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760721.890440:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.890446:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.890459:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.942648:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971694 is committed 00002000:00010000:3.0:1789760721.942818:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17915904 -> 17920000 00010000:00010000:3.0:1789760721.942832:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942845:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942861:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942868:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760721.942872:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942880:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942898:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.942905:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.943043:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507188096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52688us (52951us total) trans 4294971694 rc 0/0 00000100:00100000:3.0:1789760721.943052:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4404 00000100:00100000:3.0:1789760721.943062:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703144307361 time 703205641604 state 0x2 ref 0 00000100:00100000:2.0:1789760721.949170:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760721.949303:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188224 00000100:00100000:0.0:1789760721.949312:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188224 00000100:00100000:0.0:1789760721.949367:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703144307361 time 703205641604 state 0x0 ref 1 00000100:00100000:0.0:1789760721.949450:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703205902020 time 703205641604 state 0x1 ref 1 00000100:00100000:0.0:1789760721.949523:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703205902020 time 703264825801 state 0x0 ref 1 00000100:00100000:0.0:1789760721.949536:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507188224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760721.949594:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17920000, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:0.0:1789760721.949605:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.949623:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760721.949627:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.949635:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.949650:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760721.987215:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17920000 -> 17924096 00010000:00010000:0.0:1789760721.987237:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987251:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987260:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987273:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760721.987278:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987287:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987306:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987313:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b73dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.987398:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971695 is committed 00000100:00100000:0.0:1789760721.987417:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507188224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37878us (38250us total) trans 4294971695 rc 0/0 00000100:00100000:0.0:1789760721.987429:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4405 00000100:00100000:0.0:1789760721.987439:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703205902020 time 703264825801 state 0x2 ref 0 00000100:00100000:0.0:1789760721.992581:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760721.992667:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188352 00000100:00100000:3.0:1789760721.992678:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188352 00000100:00100000:3.0:1789760721.992774:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703205902020 time 703264825801 state 0x0 ref 1 00000100:00100000:3.0:1789760721.992792:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703265086217 time 703264825801 state 0x1 ref 1 00000100:00100000:2.0:1789760721.992886:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703265086217 time 703308188016 state 0x0 ref 1 00000100:00100000:2.0:1789760721.992901:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507188352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760721.993104:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17924096, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9007 00010000:00010000:2.0:1789760721.993114:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.993130:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760721.993135:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.993141:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.993245:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.038214:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971696 is committed 00002000:00010000:1.0:1789760722.038609:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17924096 -> 17928192 00002000:00010000:1.0:1789760722.038614:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:1.0:1789760722.038617:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:1.0:1789760722.038630:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038643:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038653:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038660:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760722.038759:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038772:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038793:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.038801:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b73e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17924096->17928191] (req 17924096->17928191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.038960:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507188352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46055us (46382us total) trans 4294971696 rc 0/0 00000100:00100000:1.0:1789760722.038980:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4406 00000100:00100000:1.0:1789760722.038999:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703265086217 time 703308188016 state 0x2 ref 0 00000100:00100000:1.0:1789760722.045810:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.045925:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188480 00000100:00100000:3.0:1789760722.046158:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188480 00000100:00100000:3.0:1789760722.046212:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703265086217 time 703308188016 state 0x0 ref 1 00000100:00100000:3.0:1789760722.046228:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703308448432 time 703308188016 state 0x1 ref 1 00000100:00100000:3.0:1789760722.046259:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703308448432 time 703361562646 state 0x0 ref 1 00000100:00100000:3.0:1789760722.046271:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507188480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.046559:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17928192, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.046569:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.046585:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.046589:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.046595:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.046608:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.099216:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971697 is committed 00002000:00010000:3.0:1789760722.099474:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17928192 -> 17932288 00010000:00010000:3.0:1789760722.099490:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099501:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099511:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099518:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.099523:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099533:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099572:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.099582:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.099919:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507188480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53643us (54113us total) trans 4294971697 rc 0/0 00000100:00100000:3.0:1789760722.099930:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4407 00000100:00100000:3.0:1789760722.102777:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703308448432 time 703361562646 state 0x2 ref 0 00000100:00100000:2.0:1789760722.105239:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760722.105533:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188608 00000100:00100000:0.0:1789760722.105543:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188608 00000100:00100000:0.0:1789760722.105598:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703308448432 time 703361562646 state 0x0 ref 1 00000100:00100000:0.0:1789760722.105614:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703361823062 time 703361562646 state 0x1 ref 1 00000100:00100000:0.0:1789760722.105643:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703361823062 time 703420946954 state 0x0 ref 1 00000100:00100000:0.0:1789760722.105656:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507188608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760722.105752:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17932288, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:0.0:1789760722.105761:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.105778:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760722.105782:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.105789:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.105804:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.164471:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971698 is committed 00002000:00010000:3.0:1789760722.165824:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17932288 -> 17936384 00010000:00010000:3.0:1789760722.165844:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165865:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165876:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165884:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.165889:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165898:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165919:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.165927:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b73f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17932288->17936383] (req 17932288->17936383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.166056:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507188608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60397us (60820us total) trans 4294971698 rc 0/0 00000100:00100000:3.0:1789760722.166067:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4408 00000100:00100000:3.0:1789760722.166075:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703361823062 time 703420946954 state 0x2 ref 0 00000100:00100000:0.0:1789760722.174059:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760722.181715:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188736 00000100:00100000:1.0:1789760722.181732:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188736 00000100:00100000:2.0:1789760722.181889:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703361823062 time 703420946954 state 0x0 ref 1 00000100:00100000:2.0:1789760722.181905:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703421207370 time 703420946954 state 0x1 ref 1 00000100:00100000:2.0:1789760722.181948:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703421207370 time 703497251970 state 0x0 ref 1 00000100:00100000:2.0:1789760722.181961:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507188736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760722.182012:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17936384, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:2.0:1789760722.182021:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.182037:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760722.182042:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.182048:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.182060:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.276997:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971699 is committed 00002000:00010000:3.0:1789760722.277166:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17936384 -> 17940480 00010000:00010000:3.0:1789760722.277182:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277193:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277202:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277209:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.277213:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277221:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277239:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.277247:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b73f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17936384->17940479] (req 17936384->17940479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.277334:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507188736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 95370us (103280us total) trans 4294971699 rc 0/0 00000100:00100000:3.0:1789760722.277344:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4409 00000100:00100000:3.0:1789760722.277353:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703421207370 time 703497251970 state 0x2 ref 0 00000100:00100000:1.0:1789760722.282605:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.284062:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188864 00000100:00100000:3.0:1789760722.284075:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188864 00000100:00100000:3.0:1789760722.284138:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703421207370 time 703497251970 state 0x0 ref 1 00000100:00100000:3.0:1789760722.284763:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703497512386 time 703497251970 state 0x1 ref 1 00000100:00100000:3.0:1789760722.284800:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703497512386 time 703600102638 state 0x0 ref 1 00000100:00100000:3.0:1789760722.284814:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507188864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.284864:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17940480, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.284872:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.284887:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.284891:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.284897:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.284911:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.357234:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971700 is committed 00002000:00010000:3.0:1789760722.357459:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17940480 -> 17944576 00010000:00010000:3.0:1789760722.357473:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357488:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357498:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357505:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.357510:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357520:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357539:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.357547:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b73ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17940480->17944575] (req 17940480->17944575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.357638:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507188864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72824us (75047us total) trans 4294971700 rc 0/0 00000100:00100000:3.0:1789760722.357648:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4410 00000100:00100000:3.0:1789760722.357658:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703497512386 time 703600102638 state 0x2 ref 0 00000100:00100000:2.0:1789760722.366231:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.366330:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507188992 00000100:00100000:3.0:1789760722.366341:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507188992 00000100:00100000:3.0:1789760722.366397:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703497512386 time 703600102638 state 0x0 ref 1 00000100:00100000:3.0:1789760722.366412:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703600363054 time 703600102638 state 0x1 ref 1 00000100:00100000:3.0:1789760722.366446:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703600363054 time 703681749682 state 0x0 ref 1 00000100:00100000:3.0:1789760722.366458:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507188992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.366508:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17944576, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.366517:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.366532:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.366537:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.366543:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.366555:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760722.405113:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17944576 -> 17948672 00010000:00010000:3.0:1789760722.405131:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405145:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405154:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405162:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.405167:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405176:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405194:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.405202:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7406 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.405295:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507188992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38833us (39068us total) trans 4294971701 rc 0/0 00000100:00100000:3.0:1789760722.405305:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4411 00000100:00100000:3.0:1789760722.405314:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703600363054 time 703681749682 state 0x2 ref 0 00000001:00080000:1.0:1789760722.405385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971701 is committed 00000100:00100000:2.0:1789760722.416658:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.416778:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189120 00000100:00100000:3.0:1789760722.416787:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189120 00000100:00100000:3.0:1789760722.416843:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703600363054 time 703681749682 state 0x0 ref 1 00000100:00100000:3.0:1789760722.416859:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703682010098 time 703681749682 state 0x1 ref 1 00000100:00100000:3.0:1789760722.416889:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703682010098 time 703732192694 state 0x0 ref 1 00000100:00100000:3.0:1789760722.416903:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507189120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.417133:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17948672, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.417143:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.417158:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.417163:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.417168:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.417182:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.510937:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971702 is committed 00002000:00010000:3.0:1789760722.511136:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17948672 -> 17952768 00010000:00010000:3.0:1789760722.511152:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511164:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511173:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511180:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.511185:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511193:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511217:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.511225:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b740d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.511318:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507189120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 94412us (94665us total) trans 4294971702 rc 0/0 00000100:00100000:3.0:1789760722.511328:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4412 00000100:00100000:3.0:1789760722.511337:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703682010098 time 703732192694 state 0x2 ref 0 00000100:00100000:0.0:1789760722.521148:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.521232:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189248 00000100:00100000:3.0:1789760722.521242:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189248 00000100:00100000:3.0:1789760722.521304:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703682010098 time 703732192694 state 0x0 ref 1 00000100:00100000:3.0:1789760722.521320:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703732453110 time 703732192694 state 0x1 ref 1 00000100:00100000:3.0:1789760722.521349:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703732453110 time 703836652117 state 0x0 ref 1 00000100:00100000:3.0:1789760722.521363:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507189248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.521410:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17952768, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.521419:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.521433:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.521438:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.521443:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.521457:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.607489:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971703 is committed 00002000:00010000:3.0:1789760722.607588:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17952768 -> 17956864 00010000:00010000:3.0:1789760722.607603:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607614:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607623:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607630:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.607634:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607642:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607662:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.607674:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7414 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.607817:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507189248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 86452us (86674us total) trans 4294971703 rc 0/0 00000100:00100000:3.0:1789760722.607827:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4413 00000100:00100000:3.0:1789760722.607837:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703732453110 time 703836652117 state 0x2 ref 0 00000100:00100000:2.0:1789760722.611553:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.611623:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189376 00000100:00100000:3.0:1789760722.611633:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189376 00000100:00100000:3.0:1789760722.611684:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703732453110 time 703836652117 state 0x0 ref 1 00000100:00100000:3.0:1789760722.611726:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703836912533 time 703836652117 state 0x1 ref 1 00000100:00100000:3.0:1789760722.611751:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703836912533 time 703927054159 state 0x0 ref 1 00000100:00100000:3.0:1789760722.611761:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507189376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.611805:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17956864, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.611813:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.611828:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760722.611833:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.611838:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.611856:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.652778:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.652810:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.652841:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.652860:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.652882:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.652919:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189632 00000100:00100000:2.0:1789760722.652937:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189632 00000100:00100000:3.0:1789760722.653633:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189760 00000100:00100000:3.0:1789760722.653640:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189760 00000100:00100000:3.0:1789760722.653714:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189888 00000100:00100000:3.0:1789760722.653717:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189888 00000100:00100000:3.0:1789760722.653733:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190016 00000100:00100000:3.0:1789760722.653736:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190016 00000100:00100000:3.0:1789760722.653757:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 848 00000100:00100000:3.0:1789760722.653766:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507189760:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760722.653840:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507189760:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 74us (997us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.653862:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 848 00000100:00100000:3.0:1789760722.653880:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 849 00000100:00100000:3.0:1789760722.653887:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507189888:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760722.653926:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507189888:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 39us (1064us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.653964:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 849 00000100:00100000:3.0:1789760722.654287:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 850 00000100:00100000:3.0:1789760722.654294:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507190016:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760722.654321:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507190016:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 28us (1438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.654327:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 850 00000100:00080000:3.0:1789760722.654354:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760722.654359:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760722.655539:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507189504 00000100:00100000:1.0:1789760722.655550:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507189504 00000100:00100000:1.0:1789760722.655602:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 460 00000100:00100000:1.0:1789760722.655610:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223495050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507189504:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760722.655664:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223495050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507189504:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 55us (2896us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760722.655671:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 460 00000100:00100000:2.0:1789760722.656360:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 851 00000100:00100000:2.0:1789760722.656370:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507189632:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760722.656422:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507189632:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 53us (3611us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.656428:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 851 00000100:00080000:1.0:1789760722.658096:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760722.702162:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971704 is committed 00002000:00010000:3.0:1789760722.702237:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17956864 -> 17960960 00010000:00010000:3.0:1789760722.702252:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702263:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702272:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702278:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.702283:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702291:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702311:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.702319:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b741b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.702406:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507189376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 90641us (90857us total) trans 4294971704 rc 0/0 00000100:00100000:3.0:1789760722.702417:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4414 00000100:00100000:3.0:1789760722.702427:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703836912533 time 703927054159 state 0x2 ref 0 00000100:00100000:1.0:1789760722.711519:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.711951:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190144 00000100:00100000:3.0:1789760722.711962:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190144 00000100:00100000:3.0:1789760722.712018:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703836912533 time 703927054159 state 0x0 ref 1 00000100:00100000:3.0:1789760722.712034:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703927314575 time 703927054159 state 0x1 ref 1 00000100:00100000:3.0:1789760722.712065:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703927314575 time 704027368668 state 0x0 ref 1 00000100:00100000:3.0:1789760722.712077:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21faccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.712127:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17960960, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.712136:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.712151:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760722.712155:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.712161:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.712175:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.733591:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971705 is committed 00002000:00010000:3.0:1789760722.733614:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17960960 -> 17965056 00010000:00010000:3.0:1789760722.733629:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733648:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733657:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733662:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.733666:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733671:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733687:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.733707:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7422 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.733782:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21faccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21703us (22268us total) trans 4294971705 rc 0/0 00000100:00100000:3.0:1789760722.733792:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4415 00000100:00100000:3.0:1789760722.733802:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703927314575 time 704027368668 state 0x2 ref 0 00000100:00100000:2.0:1789760722.740913:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.741349:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190272 00000100:00100000:3.0:1789760722.741359:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190272 00000100:00100000:3.0:1789760722.741416:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 703927314575 time 704027368668 state 0x0 ref 1 00000100:00100000:3.0:1789760722.741429:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704027629084 time 704027368668 state 0x1 ref 1 00000100:00100000:3.0:1789760722.741453:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704027629084 time 704056756566 state 0x0 ref 1 00000100:00100000:3.0:1789760722.741465:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.741580:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17965056, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.741591:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.741607:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760722.741611:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.741618:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.741631:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.763477:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971706 is committed 00002000:00010000:3.0:1789760722.763600:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17965056 -> 17969152 00010000:00010000:3.0:1789760722.763615:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763628:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763641:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763648:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.763653:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763662:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763683:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.763718:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7429 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.764015:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22546us (23108us total) trans 4294971706 rc 0/0 00000100:00100000:3.0:1789760722.764025:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4416 00000100:00100000:3.0:1789760722.764034:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704027629084 time 704056756566 state 0x2 ref 0 00000100:00100000:2.0:1789760722.769080:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760722.769192:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190400 00000100:00100000:0.0:1789760722.769202:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190400 00000100:00100000:0.0:1789760722.769258:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704027629084 time 704056756566 state 0x0 ref 1 00000100:00100000:0.0:1789760722.769338:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704057016982 time 704056756566 state 0x1 ref 1 00000100:00100000:0.0:1789760722.769414:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704057016982 time 704084715760 state 0x0 ref 1 00000100:00100000:0.0:1789760722.769426:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760722.769477:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17969152, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:0.0:1789760722.769485:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.769500:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760722.769505:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.769512:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.769526:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.795987:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971707 is committed 00002000:00010000:3.0:1789760722.796504:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17969152 -> 17973248 00010000:00010000:3.0:1789760722.796519:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796533:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796539:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796544:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.796548:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796553:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796568:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.796573:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7430 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.796647:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27218us (27572us total) trans 4294971707 rc 0/0 00000100:00100000:3.0:1789760722.796657:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4417 00000100:00100000:3.0:1789760722.796667:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704057016982 time 704084715760 state 0x2 ref 0 00000100:00100000:1.0:1789760722.801036:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.802550:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190528 00000100:00100000:2.0:1789760722.802564:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190528 00000100:00100000:2.0:1789760722.802625:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704057016982 time 704084715760 state 0x0 ref 1 00000100:00100000:2.0:1789760722.802641:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704084976176 time 704084715760 state 0x1 ref 1 00000100:00100000:2.0:1789760722.802676:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704084976176 time 704117979066 state 0x0 ref 1 00000100:00100000:2.0:1789760722.802689:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760722.802765:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17973248, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:2.0:1789760722.802774:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.802791:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760722.802796:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.802802:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.802817:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.831758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971708 is committed 00002000:00010000:3.0:1789760722.831887:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17973248 -> 17977344 00010000:00010000:3.0:1789760722.831948:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.831961:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.832037:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.832045:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760722.832050:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.832060:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.832079:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.832085:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7437 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.832212:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29521us (31181us total) trans 4294971708 rc 0/0 00000100:00100000:3.0:1789760722.832222:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4418 00000100:00100000:3.0:1789760722.832232:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704084976176 time 704117979066 state 0x2 ref 0 00000100:00100000:2.0:1789760722.836249:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.836317:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190656 00000100:00100000:3.0:1789760722.836327:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190656 00000100:00100000:3.0:1789760722.836447:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704084976176 time 704117979066 state 0x0 ref 1 00000100:00100000:3.0:1789760722.836464:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704118239482 time 704117979066 state 0x1 ref 1 00000100:00100000:3.0:1789760722.836496:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704118239482 time 704151798708 state 0x0 ref 1 00000100:00100000:3.0:1789760722.836508:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.836556:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17977344, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.836565:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.836580:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760722.836584:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.836590:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.836604:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.844578:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760722.861741:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971709 is committed 00002000:00010000:2.0:1789760722.861817:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17977344 -> 17981440 00010000:00010000:2.0:1789760722.861832:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861845:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861855:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861863:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760722.861868:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861876:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861896:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.861904:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b743e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.862013:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507190656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25502us (25770us total) trans 4294971709 rc 0/0 00000100:00100000:2.0:1789760722.862023:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4419 00000100:00100000:2.0:1789760722.862033:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704118239482 time 704151798708 state 0x2 ref 0 00000100:00100000:2.0:1789760722.864914:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.865134:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190784 00000100:00100000:3.0:1789760722.865143:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190784 00000100:00100000:3.0:1789760722.865197:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704118239482 time 704151798708 state 0x0 ref 1 00000100:00100000:3.0:1789760722.865211:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704152059124 time 704151798708 state 0x1 ref 1 00000100:00100000:3.0:1789760722.865241:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704152059124 time 704180544460 state 0x0 ref 1 00000100:00100000:3.0:1789760722.865253:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507190784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.865302:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17981440, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.865311:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.865327:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760722.865331:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.865337:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.865350:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.889593:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17981440 -> 17985536 00010000:00010000:2.0:1789760722.889612:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889625:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889636:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889645:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760722.889649:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889658:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889677:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.889722:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7445 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17981440->17985535] (req 17981440->17985535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.889892:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507190784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24635us (24982us total) trans 4294971710 rc 0/0 00000100:00100000:2.0:1789760722.889903:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4420 00000100:00100000:2.0:1789760722.889913:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704152059124 time 704180544460 state 0x2 ref 0 00000001:00080000:1.0:1789760722.890025:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971710 is committed 00000100:00100000:1.0:1789760722.893847:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760722.893938:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507190912 00000100:00100000:3.0:1789760722.893948:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507190912 00000100:00100000:3.0:1789760722.894005:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704152059124 time 704180544460 state 0x0 ref 1 00000100:00100000:3.0:1789760722.894022:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704180804876 time 704180544460 state 0x1 ref 1 00000100:00100000:3.0:1789760722.894051:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704180804876 time 704209354144 state 0x0 ref 1 00000100:00100000:3.0:1789760722.894064:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760722.894117:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17985536, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:3.0:1789760722.894127:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.894144:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760722.894148:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.894155:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.894168:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.939848:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971711 is committed 00002000:00010000:2.0:1789760722.939943:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17985536 -> 17989632 00010000:00010000:2.0:1789760722.939959:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940041:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940052:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940060:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760722.940064:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940075:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940101:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.940109:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b744c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.941167:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507190912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47094us (47318us total) trans 4294971711 rc 0/0 00000100:00100000:2.0:1789760722.941180:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4421 00000100:00100000:2.0:1789760722.941243:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704180804876 time 704209354144 state 0x2 ref 0 00000100:00100000:3.0:1789760722.946015:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.946126:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191040 00000100:00100000:2.0:1789760722.946136:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191040 00000100:00100000:2.0:1789760722.946191:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704180804876 time 704209354144 state 0x0 ref 1 00000100:00100000:2.0:1789760722.946207:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704209614560 time 704209354144 state 0x1 ref 1 00000100:00100000:2.0:1789760722.946228:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704209614560 time 704261531047 state 0x0 ref 1 00000100:00100000:2.0:1789760722.946239:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507191040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760722.946286:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17989632, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:2.0:1789760722.946295:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.946311:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760722.946316:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.946323:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.946337:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.978889:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971712 is committed 00002000:00010000:2.0:1789760722.979086:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17989632 -> 17993728 00010000:00010000:2.0:1789760722.979102:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979114:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979124:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979131:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760722.979136:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979145:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979164:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.979176:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7453 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17989632->17993727] (req 17989632->17993727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.979267:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507191040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33025us (33259us total) trans 4294971712 rc 0/0 00000100:00100000:2.0:1789760722.979277:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4422 00000100:00100000:2.0:1789760722.979287:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704209614560 time 704261531047 state 0x2 ref 0 00000100:00100000:3.0:1789760722.988393:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760722.988487:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191168 00000100:00100000:2.0:1789760722.988498:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191168 00000100:00100000:2.0:1789760722.988564:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704209614560 time 704261531047 state 0x0 ref 1 00000100:00100000:2.0:1789760722.988579:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704261791463 time 704261531047 state 0x1 ref 1 00000100:00100000:2.0:1789760722.988611:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704261791463 time 704303914112 state 0x0 ref 1 00000100:00100000:2.0:1789760722.988624:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760722.988673:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17993728, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9007 00010000:00010000:2.0:1789760722.988682:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.988725:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760722.988731:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.988737:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.988758:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.041051:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971713 is committed 00002000:00010000:3.0:1789760723.041205:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17993728 -> 17997824 00002000:00010000:3.0:1789760723.041210:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:3.0:1789760723.041213:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:3.0:1789760723.041226:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041239:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041249:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041256:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760723.041261:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041272:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041291:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.041299:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b745a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17993728->17997823] (req 17993728->17997823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.041392:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52765us (53004us total) trans 4294971713 rc 0/0 00000100:00100000:3.0:1789760723.041403:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4423 00000100:00100000:3.0:1789760723.041412:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704261791463 time 704303914112 state 0x2 ref 0 00000100:00100000:1.0:1789760723.047532:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.047719:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191296 00000100:00100000:2.0:1789760723.047729:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191296 00000100:00100000:2.0:1789760723.047832:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704261791463 time 704303914112 state 0x0 ref 1 00000100:00100000:2.0:1789760723.047850:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704304174528 time 704303914112 state 0x1 ref 1 00000100:00100000:2.0:1789760723.047881:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704304174528 time 704363184343 state 0x0 ref 1 00000100:00100000:2.0:1789760723.047894:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507191296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.047956:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17997824, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.047975:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.047991:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.047996:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.048002:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.048015:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760723.049057:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760723.072600:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971714 is committed 00002000:00010000:3.0:1789760723.072714:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17997824 -> 18001920 00010000:00010000:3.0:1789760723.072729:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072742:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072752:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072759:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760723.072764:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072775:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072792:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.072800:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7461 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.072880:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507191296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24985us (25354us total) trans 4294971714 rc 0/0 00000100:00100000:3.0:1789760723.072889:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4424 00000100:00100000:3.0:1789760723.072898:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704304174528 time 704363184343 state 0x2 ref 0 00000100:00100000:0.0:1789760723.076428:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760723.076503:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191424 00000100:00100000:1.0:1789760723.076511:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191424 00000100:00100000:1.0:1789760723.076561:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704304174528 time 704363184343 state 0x0 ref 1 00000100:00100000:1.0:1789760723.076575:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704363444759 time 704363184343 state 0x1 ref 1 00000100:00100000:1.0:1789760723.076603:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704363444759 time 704391906291 state 0x0 ref 1 00000100:00100000:1.0:1789760723.076617:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760723.076661:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18001920, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:1.0:1789760723.076670:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.076684:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760723.076688:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.076718:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.076732:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.103074:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18001920 -> 18006016 00010000:00010000:2.0:1789760723.103096:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103109:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103119:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103126:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.103131:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103140:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103161:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.103168:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7468 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18001920->18006015] (req 18001920->18006015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.103259:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507191424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26637us (26835us total) trans 4294971715 rc 0/0 00000100:00100000:2.0:1789760723.103271:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4425 00000100:00100000:2.0:1789760723.103280:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704363444759 time 704391906291 state 0x2 ref 0 00000001:00080000:1.0:1789760723.103333:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971715 is committed 00000100:00100000:3.0:1789760723.106304:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.106391:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191552 00000100:00100000:2.0:1789760723.106401:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191552 00000100:00100000:2.0:1789760723.106452:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704363444759 time 704391906291 state 0x0 ref 1 00000100:00100000:2.0:1789760723.106467:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704392166707 time 704391906291 state 0x1 ref 1 00000100:00100000:2.0:1789760723.106497:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704392166707 time 704421800260 state 0x0 ref 1 00000100:00100000:2.0:1789760723.106508:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.106551:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18006016, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.106560:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.106574:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.106578:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.106584:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.106598:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.141326:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971716 is committed 00002000:00010000:0.0:1789760723.141428:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18006016 -> 18010112 00010000:00010000:0.0:1789760723.141444:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141457:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141466:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141478:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760723.141482:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141490:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141510:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.141518:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b746f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18006016->18010111] (req 18006016->18010111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.141652:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35139us (35352us total) trans 4294971716 rc 0/0 00000100:00100000:0.0:1789760723.141663:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4426 00000100:00100000:0.0:1789760723.141672:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704392166707 time 704421800260 state 0x2 ref 0 00000100:00100000:2.0:1789760723.146384:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760723.146488:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191680 00000100:00100000:3.0:1789760723.146498:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191680 00000100:00100000:3.0:1789760723.146549:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704392166707 time 704421800260 state 0x0 ref 1 00000100:00100000:3.0:1789760723.146564:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704422060676 time 704421800260 state 0x1 ref 1 00000100:00100000:3.0:1789760723.146593:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704422060676 time 704461896579 state 0x0 ref 1 00000100:00100000:3.0:1789760723.146604:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760723.146651:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18010112, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:3.0:1789760723.146660:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.146675:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760723.146680:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.146685:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.146735:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.165167:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760723.165192:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538264448:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760723.165259:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760723.165326:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264448 00000100:00100000:2.0:1789760723.165333:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264448 00000100:00100000:2.0:1789760723.165507:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1789760723.165517:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538264448:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760723.165550:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760723.165589:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538264448:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760723.165596:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00080000:2.0:1789760723.165612:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760723.165898:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538264448:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760723.166491:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538264704:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166528:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166552:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538265216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166569:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166585:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538265728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166601:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166616:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1789760723.166620:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538264576:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166643:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166659:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538265088:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166678:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166706:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538265600:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166728:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166749:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538264960:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166768:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166779:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538265472:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166795:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166808:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22d5c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538265984:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166823:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166835:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1789760723.166839:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538264832:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166855:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166867:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538265344:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166883:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166895:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538265856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.166911:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.166939:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264704 00000100:00100000:0.0:1789760723.166943:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264704 00000100:00100000:0.0:1789760723.166980:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265216 00000100:00100000:0.0:1789760723.166983:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265216 00000100:00100000:0.0:1789760723.167011:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265728 00000100:00100000:0.0:1789760723.167014:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265728 00000100:00100000:0.0:1789760723.167028:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264960 00000100:00100000:0.0:1789760723.167030:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264960 00000100:00100000:0.0:1789760723.167040:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265344 00000100:00100000:0.0:1789760723.167042:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265344 00000100:00100000:0.0:1789760723.167052:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265856 00000100:00100000:0.0:1789760723.167054:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265856 00000100:00100000:0.0:1789760723.167067:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 852 00000100:00100000:0.0:1789760723.167073:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538264704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167116:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538264704:12345-0@lo:400:kworker.0 Request processed in 43us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167123:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 852 00000100:00100000:0.0:1789760723.167136:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1789760723.167141:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538265216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167168:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538265216:12345-0@lo:400:kworker.0 Request processed in 26us (596us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167173:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1789760723.167189:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1789760723.167194:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538265728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167220:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538265728:12345-0@lo:400:kworker.0 Request processed in 26us (618us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167225:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1789760723.167237:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 855 00000100:00100000:0.0:1789760723.167243:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538264960:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167272:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538264960:12345-0@lo:400:kworker.0 Request processed in 29us (503us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167278:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 855 00000100:00100000:0.0:1789760723.167289:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 856 00000100:00100000:0.0:1789760723.167295:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538265344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167321:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538265344:12345-0@lo:400:kworker.0 Request processed in 26us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167327:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 856 00000100:00100000:0.0:1789760723.167340:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 857 00000100:00100000:0.0:1789760723.167345:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538265856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167368:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538265856:12345-0@lo:400:kworker.0 Request processed in 23us (456us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167373:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 857 00000100:00100000:0.0:1789760723.167397:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264576 00000100:00100000:0.0:1789760723.167400:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264576 00000100:00100000:0.0:1789760723.167429:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1789760723.167435:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223494050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538264576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167468:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223494050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538264576:12345-0@lo:400:kworker.0 Request processed in 28us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167474:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1789760723.167503:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265088 00000100:00100000:0.0:1789760723.167507:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265088 00000100:00100000:0.0:1789760723.167532:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538264832 00000100:00100000:0.0:1789760723.167534:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538264832 00000100:00100000:0.0:1789760723.167550:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1789760723.167557:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538265088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167594:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538265088:12345-0@lo:400:kworker.0 Request processed in 38us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167600:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1789760723.167615:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1789760723.167622:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538264832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167651:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538264832:12345-0@lo:400:kworker.0 Request processed in 30us (794us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167656:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1789760723.167673:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265600 00000100:00100000:0.0:1789760723.167677:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265600 00000100:00100000:0.0:1789760723.167727:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265472 00000100:00100000:0.0:1789760723.167730:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265472 00000100:00100000:0.0:1789760723.167757:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538265984 00000100:00100000:0.0:1789760723.167760:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538265984 00000100:00100000:0.0:1789760723.167781:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1789760723.167790:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538265600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167827:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538265600:12345-0@lo:400:kworker.0 Request processed in 39us (1097us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167834:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1789760723.167849:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789760723.167856:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538265472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167890:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538265472:12345-0@lo:400:kworker.0 Request processed in 35us (1093us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167897:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789760723.167915:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789760723.167921:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538265984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.167970:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538265984:12345-0@lo:400:kworker.0 Request processed in 49us (1145us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.167976:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789760723.168044:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538264832:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168056:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538265344:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168064:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538265856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168073:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538264960:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168082:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538265472:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168091:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22d5c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538265984:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168125:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538264576:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168136:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538265088:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168146:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538265600:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168155:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538264704:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168164:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538265216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760723.168173:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538265728:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.168723:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266112:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760723.168760:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789760723.168834:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760723.168839:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760723.168841:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760723.168843:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760723.169083:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266112 00000100:00100000:3.0:1789760723.169088:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266112 00000100:00100000:3.0:1789760723.169128:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789760723.169135:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538266112:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760723.169151:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760723.169189:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538266112:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.169195:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789760723.169227:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266112:0@lo:13:osp-pre-1-1.0 00000001:00080000:2.0:1789760723.186817:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971717 is committed 00002000:00010000:1.0:1789760723.187047:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18010112 -> 18014208 00010000:00010000:1.0:1789760723.187062:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187076:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187085:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187092:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760723.187097:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187106:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187125:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.187133:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7476 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18010112->18014207] (req 18010112->18014207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.187234:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40626us (40867us total) trans 4294971717 rc 0/0 00000100:00100000:1.0:1789760723.187243:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4427 00000100:00100000:1.0:1789760723.187252:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704422060676 time 704461896579 state 0x2 ref 0 00000100:00100000:1.0:1789760723.192063:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.192277:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191808 00000100:00100000:2.0:1789760723.192354:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191808 00000100:00100000:2.0:1789760723.192410:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704422060676 time 704461896579 state 0x0 ref 1 00000100:00100000:2.0:1789760723.192427:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704462156995 time 704461896579 state 0x1 ref 1 00000100:00100000:2.0:1789760723.192459:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704462156995 time 704507761706 state 0x0 ref 1 00000100:00100000:2.0:1789760723.192472:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.192533:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18014208, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.192543:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.192559:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.192565:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.192571:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.192586:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.269385:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18014208 -> 18018304 00010000:00010000:2.0:1789760723.269408:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.269416:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971718 is committed 00010000:00010000:2.0:1789760723.269421:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.269431:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.269438:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.269443:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.269450:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.269470:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.269481:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b747d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18014208->18018303] (req 18014208->18018303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.269610:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507191808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77135us (77551us total) trans 4294971718 rc 0/0 00000100:00100000:2.0:1789760723.269621:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4428 00000100:00100000:2.0:1789760723.269632:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704462156995 time 704507761706 state 0x2 ref 0 00000100:00100000:3.0:1789760723.278352:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760723.279135:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507191936 00000100:00100000:0.0:1789760723.279156:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507191936 00000100:00100000:0.0:1789760723.279211:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704462156995 time 704507761706 state 0x0 ref 1 00000100:00100000:0.0:1789760723.279226:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704508022122 time 704507761706 state 0x1 ref 1 00000100:00100000:0.0:1789760723.279259:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704508022122 time 704594562813 state 0x0 ref 1 00000100:00100000:0.0:1789760723.279273:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507191936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760723.279323:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18018304, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:0.0:1789760723.279332:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.279348:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760723.279352:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.279358:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.279372:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.316690:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971719 is committed 00002000:00010000:2.0:1789760723.316861:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18018304 -> 18022400 00010000:00010000:2.0:1789760723.316877:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316890:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316899:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316906:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.316910:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316920:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316958:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.316976:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7484 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18018304->18022399] (req 18018304->18022399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.317074:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507191936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37799us (38728us total) trans 4294971719 rc 0/0 00000100:00100000:2.0:1789760723.317085:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4429 00000100:00100000:2.0:1789760723.317097:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704508022122 time 704594562813 state 0x2 ref 0 00000100:00100000:0.0:1789760723.323113:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760723.323570:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192064 00000100:00100000:1.0:1789760723.323581:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192064 00000100:00100000:1.0:1789760723.323637:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704508022122 time 704594562813 state 0x0 ref 1 00000100:00100000:1.0:1789760723.323651:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704594823229 time 704594562813 state 0x1 ref 1 00000100:00100000:1.0:1789760723.323682:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704594823229 time 704638985716 state 0x0 ref 1 00000100:00100000:1.0:1789760723.323720:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760723.323858:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18022400, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:0.0:1789760723.323868:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.323885:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760723.323890:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.323896:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.323910:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.373198:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971720 is committed 00002000:00010000:2.0:1789760723.374796:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18022400 -> 18026496 00010000:00010000:2.0:1789760723.374815:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374828:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374838:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374845:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.374850:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374865:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374884:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.374892:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b748b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18022400->18026495] (req 18022400->18026495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.375005:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51282us (51895us total) trans 4294971720 rc 0/0 00000100:00100000:2.0:1789760723.375019:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4430 00000100:00100000:2.0:1789760723.375028:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704594823229 time 704638985716 state 0x2 ref 0 00000100:00100000:1.0:1789760723.383028:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.383131:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192192 00000100:00100000:2.0:1789760723.383141:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192192 00000100:00100000:2.0:1789760723.383206:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704594823229 time 704638985716 state 0x0 ref 1 00000100:00100000:2.0:1789760723.383232:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704639246132 time 704638985716 state 0x1 ref 1 00000100:00100000:2.0:1789760723.383266:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704639246132 time 704698569889 state 0x0 ref 1 00000100:00100000:2.0:1789760723.383279:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.383325:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18026496, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.383335:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.383351:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.383356:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.383362:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.383375:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.432681:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971721 is committed 00002000:00010000:2.0:1789760723.432800:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18026496 -> 18030592 00010000:00010000:2.0:1789760723.432815:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432828:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432836:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432843:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.432848:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432857:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432875:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.432884:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7492 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18026496->18030591] (req 18026496->18030591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.433329:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50045us (50302us total) trans 4294971721 rc 0/0 00000100:00100000:2.0:1789760723.433342:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4431 00000100:00100000:2.0:1789760723.433352:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704639246132 time 704698569889 state 0x2 ref 0 00000100:00100000:0.0:1789760723.438269:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.438352:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192320 00000100:00100000:2.0:1789760723.438362:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192320 00000100:00100000:2.0:1789760723.440038:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704639246132 time 704698569889 state 0x0 ref 1 00000100:00100000:2.0:1789760723.440054:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704698830305 time 704698569889 state 0x1 ref 1 00000100:00100000:2.0:1789760723.440142:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704698830305 time 704755444012 state 0x0 ref 1 00000100:00100000:2.0:1789760723.440163:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.444203:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18030592, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.444228:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.444246:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.444251:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.444258:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.444280:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.482611:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971722 is committed 00002000:00010000:2.0:1789760723.482776:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18030592 -> 18034688 00010000:00010000:2.0:1789760723.482792:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482805:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482814:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482821:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.482826:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482836:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482866:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482874:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7499 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18030592->18034687] (req 18030592->18034687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.483002:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42844us (44737us total) trans 4294971722 rc 0/0 00000100:00100000:2.0:1789760723.483012:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4432 00000100:00100000:2.0:1789760723.483021:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704698830305 time 704755444012 state 0x2 ref 0 00000100:00100000:2.0:1789760723.486847:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760723.486934:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192448 00000100:00100000:0.0:1789760723.486944:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192448 00000100:00100000:0.0:1789760723.486996:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704698830305 time 704755444012 state 0x0 ref 1 00000100:00100000:0.0:1789760723.487014:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704755704428 time 704755444012 state 0x1 ref 1 00000100:00100000:0.0:1789760723.487045:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704755704428 time 704802348012 state 0x0 ref 1 00000100:00100000:0.0:1789760723.487056:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760723.487102:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18034688, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:0.0:1789760723.487112:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.487127:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760723.487131:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.487138:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.487152:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.510226:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971723 is committed 00002000:00010000:1.0:1789760723.510288:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18034688 -> 18038784 00010000:00010000:1.0:1789760723.510303:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510316:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510325:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510333:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760723.510337:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510347:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510367:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.510378:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18034688->18038783] (req 18034688->18038783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.510483:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23425us (23642us total) trans 4294971723 rc 0/0 00000100:00100000:1.0:1789760723.510490:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4433 00000100:00100000:1.0:1789760723.510498:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704755704428 time 704802348012 state 0x2 ref 0 00000100:00100000:1.0:1789760723.515071:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.515420:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192576 00000100:00100000:2.0:1789760723.515429:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192576 00000100:00100000:2.0:1789760723.515485:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704755704428 time 704802348012 state 0x0 ref 1 00000100:00100000:2.0:1789760723.515500:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704802608428 time 704802348012 state 0x1 ref 1 00000100:00100000:2.0:1789760723.515614:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704802608428 time 704830915674 state 0x0 ref 1 00000100:00100000:2.0:1789760723.515628:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.515678:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18038784, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.515687:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.515719:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.515724:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.515730:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.515745:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.550199:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971724 is committed 00002000:00010000:2.0:1789760723.550300:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18038784 -> 18042880 00010000:00010000:2.0:1789760723.550316:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550329:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550339:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550346:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.550352:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550361:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550381:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.550390:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18038784->18042879] (req 18038784->18042879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.550592:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34961us (35525us total) trans 4294971724 rc 0/0 00000100:00100000:2.0:1789760723.550606:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4434 00000100:00100000:2.0:1789760723.550616:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704802608428 time 704830915674 state 0x2 ref 0 00000100:00100000:0.0:1789760723.557528:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.558516:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192704 00000100:00100000:2.0:1789760723.558528:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192704 00000100:00100000:2.0:1789760723.558588:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704802608428 time 704830915674 state 0x0 ref 1 00000100:00100000:2.0:1789760723.558604:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704831176090 time 704830915674 state 0x1 ref 1 00000100:00100000:2.0:1789760723.558628:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704831176090 time 704873931276 state 0x0 ref 1 00000100:00100000:2.0:1789760723.558642:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.558689:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18042880, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.558733:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.558749:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760723.558754:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.558761:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.558775:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.649424:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971725 is committed 00002000:00010000:2.0:1789760723.649549:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18042880 -> 18046976 00010000:00010000:2.0:1789760723.649564:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649578:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649588:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649596:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.649601:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649611:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649633:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.649641:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18042880->18046975] (req 18042880->18046975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.649763:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91119us (92242us total) trans 4294971725 rc 0/0 00000100:00100000:2.0:1789760723.649774:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4435 00000100:00100000:2.0:1789760723.649784:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704831176090 time 704873931276 state 0x2 ref 0 00000100:00100000:1.0:1789760723.654632:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760723.654724:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192832 00000100:00100000:0.0:1789760723.654733:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192832 00000100:00100000:0.0:1789760723.654788:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704831176090 time 704873931276 state 0x0 ref 1 00000100:00100000:0.0:1789760723.654804:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704874191692 time 704873931276 state 0x1 ref 1 00000100:00100000:0.0:1789760723.654836:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704874191692 time 704970138581 state 0x0 ref 1 00000100:00100000:0.0:1789760723.654854:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760723.654902:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18046976, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:0.0:1789760723.654911:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.654937:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760723.654943:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.654950:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.655082:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.698560:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971726 is committed 00002000:00010000:0.0:1789760723.698628:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18046976 -> 18051072 00010000:00010000:0.0:1789760723.698647:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698659:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698667:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698714:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760723.698719:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698727:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698748:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698756:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18046976->18051071] (req 18046976->18051071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.698840:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507192832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43983us (44212us total) trans 4294971726 rc 0/0 00000100:00100000:0.0:1789760723.698850:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4436 00000100:00100000:0.0:1789760723.698860:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704874191692 time 704970138581 state 0x2 ref 0 00000100:00100000:1.0:1789760723.705543:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760723.705630:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507192960 00000100:00100000:3.0:1789760723.705640:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507192960 00000100:00100000:3.0:1789760723.705724:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704874191692 time 704970138581 state 0x0 ref 1 00000100:00100000:3.0:1789760723.705740:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704970398997 time 704970138581 state 0x1 ref 1 00000100:00100000:3.0:1789760723.705765:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704970398997 time 705021068740 state 0x0 ref 1 00000100:00100000:3.0:1789760723.705777:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507192960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760723.705826:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18051072, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:3.0:1789760723.705834:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.705848:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760723.705853:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.705860:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.705873:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760723.763056:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18051072 -> 18055168 00010000:00010000:3.0:1789760723.763075:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763086:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763096:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763103:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760723.763108:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763117:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763134:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.763141:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b74bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18051072->18055167] (req 18051072->18055167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.763225:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507192960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57445us (57687us total) trans 4294971727 rc 0/0 00000100:00100000:3.0:1789760723.763235:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4437 00000100:00100000:3.0:1789760723.763244:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704970398997 time 705021068740 state 0x2 ref 0 00000001:00080000:0.0:1789760723.763673:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971727 is committed 00000100:00100000:0.0:1789760723.767892:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.768044:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193088 00000100:00100000:2.0:1789760723.768054:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193088 00000100:00100000:2.0:1789760723.768107:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 704970398997 time 705021068740 state 0x0 ref 1 00000100:00100000:2.0:1789760723.768123:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705021329156 time 705021068740 state 0x1 ref 1 00000100:00100000:2.0:1789760723.768150:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705021329156 time 705083453669 state 0x0 ref 1 00000100:00100000:2.0:1789760723.768163:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.768207:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18055168, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.768216:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.768232:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760723.768236:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.768242:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.768255:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760723.823205:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18055168 -> 18059264 00010000:00010000:0.0:1789760723.823226:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823239:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823248:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823262:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760723.823267:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823277:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823297:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.823306:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b74c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.823395:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507193088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55230us (55510us total) trans 4294971728 rc 0/0 00000100:00100000:0.0:1789760723.823406:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4438 00000100:00100000:0.0:1789760723.823415:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705021329156 time 705083453669 state 0x2 ref 0 00000001:00080000:3.0:1789760723.824086:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971728 is committed 00000100:00100000:3.0:1789760723.837359:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760723.837584:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193216 00000100:00100000:0.0:1789760723.837594:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193216 00000100:00100000:0.0:1789760723.837740:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705021329156 time 705083453669 state 0x0 ref 1 00000100:00100000:0.0:1789760723.837757:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705083714085 time 705083453669 state 0x1 ref 1 00000100:00100000:0.0:1789760723.837840:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705083714085 time 705153142675 state 0x0 ref 1 00000100:00100000:0.0:1789760723.837857:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507193216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760723.837905:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18059264, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:0.0:1789760723.837927:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.837942:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760723.837947:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.837953:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.837967:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.867890:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760723.883021:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971729 is committed 00002000:00010000:2.0:1789760723.883189:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18059264 -> 18063360 00010000:00010000:2.0:1789760723.883206:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883216:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883229:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883236:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.883241:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883248:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883334:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.883344:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b74ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18059264->18063359] (req 18059264->18063359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.883539:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507193216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45678us (46183us total) trans 4294971729 rc 0/0 00000100:00100000:2.0:1789760723.883551:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4439 00000100:00100000:2.0:1789760723.883560:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705083714085 time 705153142675 state 0x2 ref 0 00000100:00100000:1.0:1789760723.888407:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760723.888492:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193344 00000100:00100000:3.0:1789760723.888502:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193344 00000100:00100000:3.0:1789760723.888552:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705083714085 time 705153142675 state 0x0 ref 1 00000100:00100000:3.0:1789760723.888567:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705153403091 time 705153142675 state 0x1 ref 1 00000100:00100000:3.0:1789760723.888609:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705153403091 time 705203911933 state 0x0 ref 1 00000100:00100000:3.0:1789760723.888622:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760723.888671:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18063360, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:3.0:1789760723.888680:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.888724:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760723.888728:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.888734:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.888757:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.950508:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18063360 -> 18067456 00010000:00010000:1.0:1789760723.950529:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.950530:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971730 is committed 00010000:00010000:1.0:1789760723.950544:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.950554:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.950562:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760723.950568:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.950577:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.950598:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.950607:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b74d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.950890:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62074us (62297us total) trans 4294971730 rc 0/0 00000100:00100000:1.0:1789760723.950901:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4440 00000100:00100000:1.0:1789760723.950911:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705153403091 time 705203911933 state 0x2 ref 0 00000100:00100000:0.0:1789760723.956385:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760723.956547:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193472 00000100:00100000:2.0:1789760723.956558:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193472 00000100:00100000:2.0:1789760723.956607:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705153403091 time 705203911933 state 0x0 ref 1 00000100:00100000:2.0:1789760723.956622:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705204172349 time 705203911933 state 0x1 ref 1 00000100:00100000:2.0:1789760723.956687:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705204172349 time 705271990177 state 0x0 ref 1 00000100:00100000:2.0:1789760723.956731:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507193472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760723.956776:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18067456, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:2.0:1789760723.956784:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.956808:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760723.956812:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.956818:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.956831:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.993488:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971731 is committed 00002000:00010000:2.0:1789760723.993571:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18067456 -> 18071552 00010000:00010000:2.0:1789760723.993587:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993600:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993610:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993617:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760723.993623:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993632:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993651:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.993657:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.993974:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507193472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37240us (37593us total) trans 4294971731 rc 0/0 00000100:00100000:2.0:1789760723.993985:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4441 00000100:00100000:2.0:1789760723.993995:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705204172349 time 705271990177 state 0x2 ref 0 00000100:00100000:2.0:1789760724.001852:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760724.002343:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193600 00000100:00100000:3.0:1789760724.002354:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193600 00000100:00100000:3.0:1789760724.002416:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705204172349 time 705271990177 state 0x0 ref 1 00000100:00100000:3.0:1789760724.002431:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705272250593 time 705271990177 state 0x1 ref 1 00000100:00100000:3.0:1789760724.002460:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705272250593 time 705317763487 state 0x0 ref 1 00000100:00100000:3.0:1789760724.002472:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760724.002521:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18071552, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9007 00010000:00010000:3.0:1789760724.002530:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.002544:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760724.002549:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.002555:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.002569:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.047172:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971732 is committed 00002000:00010000:3.0:1789760724.047310:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18071552 -> 18075648 00002000:00010000:3.0:1789760724.047316:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:3.0:1789760724.047319:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:3.0:1789760724.047332:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047347:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047357:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047365:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.047369:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047379:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047398:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047655:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b74df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18071552->18075647] (req 18071552->18075647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.048321:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45843us (46470us total) trans 4294971732 rc 0/0 00000100:00100000:2.0:1789760724.048334:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4442 00000100:00100000:2.0:1789760724.048344:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705272250593 time 705317763487 state 0x2 ref 0 00000100:00100000:2.0:1789760724.052045:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760724.052136:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193728 00000100:00100000:1.0:1789760724.052147:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193728 00000100:00100000:1.0:1789760724.052197:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705272250593 time 705317763487 state 0x0 ref 1 00000100:00100000:1.0:1789760724.052212:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705318023903 time 705317763487 state 0x1 ref 1 00000100:00100000:1.0:1789760724.052242:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705318023903 time 705367544771 state 0x0 ref 1 00000100:00100000:1.0:1789760724.052255:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507193728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760724.052301:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18075648, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:1.0:1789760724.052309:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.052324:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760724.052328:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.052334:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.052346:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.103984:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971733 is committed 00002000:00010000:0.0:1789760724.104155:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18075648 -> 18079744 00010000:00010000:0.0:1789760724.104170:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104183:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104193:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104205:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760724.104211:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104219:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104238:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.104245:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b74e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18075648->18079743] (req 18075648->18079743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.104368:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507193728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52111us (52327us total) trans 4294971733 rc 0/0 00000100:00100000:0.0:1789760724.104379:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4443 00000100:00100000:0.0:1789760724.104388:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705318023903 time 705367544771 state 0x2 ref 0 00000100:00100000:0.0:1789760724.111331:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.111440:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193856 00000100:00100000:2.0:1789760724.111448:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193856 00000100:00100000:2.0:1789760724.111503:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705318023903 time 705367544771 state 0x0 ref 1 00000100:00100000:2.0:1789760724.111516:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705367805187 time 705367544771 state 0x1 ref 1 00000100:00100000:2.0:1789760724.111574:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705367805187 time 705426877136 state 0x0 ref 1 00000100:00100000:2.0:1789760724.111595:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507193856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.111643:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18079744, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.111652:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.111667:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760724.111671:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.111677:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.111720:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.137423:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18079744 -> 18083840 00010000:00010000:2.0:1789760724.137442:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137456:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137465:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137473:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760724.137478:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137487:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137506:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.137513:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18079744->18083839] (req 18079744->18083839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.137614:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507193856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26026us (26288us total) trans 4294971734 rc 0/0 00000100:00100000:2.0:1789760724.137624:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4444 00000100:00100000:2.0:1789760724.137633:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705367805187 time 705426877136 state 0x2 ref 0 00000001:00080000:3.0:1789760724.138170:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971734 is committed 00000100:00100000:1.0:1789760724.143414:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.145374:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507193984 00000100:00100000:2.0:1789760724.145390:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507193984 00000100:00100000:2.0:1789760724.145454:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705367805187 time 705426877136 state 0x0 ref 1 00000100:00100000:2.0:1789760724.145476:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705427137552 time 705426877136 state 0x1 ref 1 00000100:00100000:2.0:1789760724.145551:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705427137552 time 705460854106 state 0x0 ref 1 00000100:00100000:2.0:1789760724.145568:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507193984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.145688:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18083840, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.145718:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.145733:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760724.145737:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.145743:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.145756:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.176551:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18083840 -> 18087936 00010000:00010000:0.0:1789760724.176569:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176591:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176607:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176615:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760724.176620:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176629:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176646:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.176654:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b74f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18083840->18087935] (req 18083840->18087935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.176761:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507193984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31192us (33352us total) trans 4294971735 rc 0/0 00000100:00100000:0.0:1789760724.176771:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4445 00000100:00100000:0.0:1789760724.176780:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705427137552 time 705460854106 state 0x2 ref 0 00000001:00080000:1.0:1789760724.177788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971735 is committed 00000100:00100000:0.0:1789760724.181073:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760724.181158:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194112 00000100:00100000:1.0:1789760724.181168:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194112 00000100:00100000:1.0:1789760724.181222:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705427137552 time 705460854106 state 0x0 ref 1 00000100:00100000:1.0:1789760724.181238:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705461114522 time 705460854106 state 0x1 ref 1 00000100:00100000:1.0:1789760724.181267:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705461114522 time 705496569708 state 0x0 ref 1 00000100:00100000:1.0:1789760724.181279:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760724.181325:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18087936, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:1.0:1789760724.181333:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.181348:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760724.181353:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.181360:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.181374:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.188739:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538266240:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760724.188804:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760724.188871:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266240 00000100:00100000:2.0:1789760724.188878:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266240 00000100:00100000:2.0:1789760724.188933:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789760724.188942:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538266240:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760724.189024:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3839885312 avail 3805282304 00000100:00100000:2.0:1789760724.189066:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538266240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 125us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760724.189073:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:3.0:1789760724.189094:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538266240:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760724.189301:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538266368:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760724.189327:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760724.189359:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266368 00000100:00100000:2.0:1789760724.189363:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266368 00000100:00100000:2.0:1789760724.189386:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1789760724.189392:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538266368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760724.189403:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3839885312 avail 3805282304 00000100:00100000:2.0:1789760724.189432:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538266368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 40us (104us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760724.189438:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1789760724.189464:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538266368:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760724.218526:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18087936 -> 18092032 00000001:00080000:3.0:1789760724.218543:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971736 is committed 00010000:00010000:2.0:1789760724.218546:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218568:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218578:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218586:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760724.218591:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218600:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218619:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.218627:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b74fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.218741:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37459us (37672us total) trans 4294971736 rc 0/0 00000100:00100000:2.0:1789760724.218750:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4446 00000100:00100000:2.0:1789760724.218760:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705461114522 time 705496569708 state 0x2 ref 0 00000100:00100000:0.0:1789760724.225463:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760724.225579:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194240 00000100:00100000:3.0:1789760724.225589:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194240 00000100:00100000:3.0:1789760724.225652:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705461114522 time 705496569708 state 0x0 ref 1 00000100:00100000:3.0:1789760724.225668:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705496830124 time 705496569708 state 0x1 ref 1 00000100:00100000:3.0:1789760724.225787:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705496830124 time 705541089131 state 0x0 ref 1 00000100:00100000:3.0:1789760724.225801:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760724.225863:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18092032, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:3.0:1789760724.225872:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.225893:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760724.225897:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.225903:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.225929:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.259575:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971737 is committed 00002000:00010000:3.0:1789760724.259616:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18092032 -> 18096128 00010000:00010000:3.0:1789760724.259632:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259645:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259655:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259663:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.259668:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259677:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259726:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.259734:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7502 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.260087:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34282us (34627us total) trans 4294971737 rc 0/0 00000100:00100000:3.0:1789760724.260099:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4447 00000100:00100000:3.0:1789760724.260110:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705496830124 time 705541089131 state 0x2 ref 0 00000100:00100000:1.0:1789760724.270186:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760724.270264:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194368 00000100:00100000:0.0:1789760724.270275:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194368 00000100:00100000:0.0:1789760724.270331:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705496830124 time 705541089131 state 0x0 ref 1 00000100:00100000:0.0:1789760724.270346:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705541349547 time 705541089131 state 0x1 ref 1 00000100:00100000:0.0:1789760724.270382:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705541349547 time 705585685230 state 0x0 ref 1 00000100:00100000:0.0:1789760724.270394:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760724.270439:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18096128, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:0.0:1789760724.270448:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.270464:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760724.270469:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.270476:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.270490:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.316569:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18096128 -> 18100224 00010000:00010000:0.0:1789760724.316590:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316604:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316620:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316628:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760724.316633:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316643:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316664:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.316672:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7509 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.316869:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507194368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46407us (46624us total) trans 4294971738 rc 0/0 00000100:00100000:0.0:1789760724.316879:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4448 00000100:00100000:0.0:1789760724.316888:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705541349547 time 705585685230 state 0x2 ref 0 00000001:00080000:3.0:1789760724.317744:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971738 is committed 00000100:00100000:0.0:1789760724.320978:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.321040:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194496 00000100:00100000:2.0:1789760724.321050:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194496 00000100:00100000:2.0:1789760724.321102:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705541349547 time 705585685230 state 0x0 ref 1 00000100:00100000:2.0:1789760724.321116:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705585945646 time 705585685230 state 0x1 ref 1 00000100:00100000:2.0:1789760724.321142:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705585945646 time 705636446271 state 0x0 ref 1 00000100:00100000:2.0:1789760724.321163:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.321213:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18100224, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.321222:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.321238:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760724.321242:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.321248:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.321269:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.367166:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971739 is committed 00002000:00010000:3.0:1789760724.367226:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18100224 -> 18104320 00010000:00010000:3.0:1789760724.367242:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367256:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367266:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367273:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.367278:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367284:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367300:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.367305:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7510 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.367469:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46303us (46495us total) trans 4294971739 rc 0/0 00000100:00100000:3.0:1789760724.367480:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4449 00000100:00100000:3.0:1789760724.367490:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705585945646 time 705636446271 state 0x2 ref 0 00000100:00100000:0.0:1789760724.371930:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760724.372019:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194624 00000100:00100000:3.0:1789760724.372028:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194624 00000100:00100000:3.0:1789760724.372082:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705585945646 time 705636446271 state 0x0 ref 1 00000100:00100000:3.0:1789760724.372097:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705636706687 time 705636446271 state 0x1 ref 1 00000100:00100000:3.0:1789760724.372131:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705636706687 time 705687433802 state 0x0 ref 1 00000100:00100000:3.0:1789760724.372144:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760724.372194:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18104320, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:3.0:1789760724.372203:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.372219:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760724.372224:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.372232:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.372246:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.404266:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971740 is committed 00002000:00010000:3.0:1789760724.404537:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18104320 -> 18108416 00010000:00010000:3.0:1789760724.404550:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404559:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404566:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404570:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.404573:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404579:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404594:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.404602:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7517 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.404687:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32542us (32772us total) trans 4294971740 rc 0/0 00000100:00100000:3.0:1789760724.404739:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4450 00000100:00100000:3.0:1789760724.404750:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705636706687 time 705687433802 state 0x2 ref 0 00000100:00100000:1.0:1789760724.410505:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760724.410682:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194752 00000100:00100000:0.0:1789760724.410734:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194752 00000100:00100000:0.0:1789760724.410794:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705636706687 time 705687433802 state 0x0 ref 1 00000100:00100000:0.0:1789760724.410810:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705687694218 time 705687433802 state 0x1 ref 1 00000100:00100000:0.0:1789760724.410849:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705687694218 time 705726152457 state 0x0 ref 1 00000100:00100000:0.0:1789760724.410862:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507194752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760724.410934:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18108416, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:0.0:1789760724.410943:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.410969:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760724.410973:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.410980:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.410993:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.449788:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18108416 -> 18112512 00010000:00010000:1.0:1789760724.449807:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449820:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449829:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449837:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760724.449841:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449851:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449871:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.449880:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b751e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.450042:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507194752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39177us (39541us total) trans 4294971741 rc 0/0 00000100:00100000:1.0:1789760724.450052:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4451 00000100:00100000:1.0:1789760724.450062:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705687694218 time 705726152457 state 0x2 ref 0 00000001:00080000:3.0:1789760724.450248:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971741 is committed 00000100:00100000:0.0:1789760724.457547:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.457661:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507194880 00000100:00100000:2.0:1789760724.457671:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507194880 00000100:00100000:2.0:1789760724.457912:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705687694218 time 705726152457 state 0x0 ref 1 00000100:00100000:2.0:1789760724.458482:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705726412873 time 705726152457 state 0x1 ref 1 00000100:00100000:2.0:1789760724.458519:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705726412873 time 705773822003 state 0x0 ref 1 00000100:00100000:2.0:1789760724.458535:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.458582:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18112512, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.458591:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.458606:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760724.458610:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.458616:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.458630:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.482622:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971742 is committed 00002000:00010000:1.0:1789760724.482740:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18112512 -> 18116608 00010000:00010000:1.0:1789760724.482757:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482774:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482783:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482790:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760724.482795:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482805:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482830:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.482838:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7525 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.482940:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507194880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24405us (25399us total) trans 4294971742 rc 0/0 00000100:00100000:1.0:1789760724.482958:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4452 00000100:00100000:1.0:1789760724.482969:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705726412873 time 705773822003 state 0x2 ref 0 00000100:00100000:0.0:1789760724.486429:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.486562:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195008 00000100:00100000:2.0:1789760724.486572:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195008 00000100:00100000:2.0:1789760724.486623:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705726412873 time 705773822003 state 0x0 ref 1 00000100:00100000:2.0:1789760724.486638:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705774082419 time 705773822003 state 0x1 ref 1 00000100:00100000:2.0:1789760724.486743:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705774082419 time 705802044846 state 0x0 ref 1 00000100:00100000:2.0:1789760724.486757:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.486802:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18116608, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.486811:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.486827:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760724.486831:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.486837:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.486850:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.547710:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971743 is committed 00002000:00010000:2.0:1789760724.548588:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18116608 -> 18120704 00010000:00010000:2.0:1789760724.548606:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548619:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548632:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548639:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760724.548644:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548653:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548674:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.548682:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b752c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.548797:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62038us (62372us total) trans 4294971743 rc 0/0 00000100:00100000:2.0:1789760724.548807:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4453 00000100:00100000:2.0:1789760724.548817:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705774082419 time 705802044846 state 0x2 ref 0 00000100:00100000:3.0:1789760724.552523:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760724.552630:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195136 00000100:00100000:1.0:1789760724.552649:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195136 00000100:00100000:1.0:1789760724.552732:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705774082419 time 705802044846 state 0x0 ref 1 00000100:00100000:1.0:1789760724.552756:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705802305262 time 705802044846 state 0x1 ref 1 00000100:00100000:1.0:1789760724.552776:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705802305262 time 705868079820 state 0x0 ref 1 00000100:00100000:1.0:1789760724.552788:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760724.552834:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18120704, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:1.0:1789760724.552842:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.552858:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760724.552863:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.552869:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.552884:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.602034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971744 is committed 00002000:00010000:2.0:1789760724.602146:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18120704 -> 18124800 00010000:00010000:2.0:1789760724.602162:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602182:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602191:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602199:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760724.602204:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602326:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602346:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.602354:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7533 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.602473:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49683us (49955us total) trans 4294971744 rc 0/0 00000100:00100000:2.0:1789760724.602484:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4454 00000100:00100000:2.0:1789760724.602493:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705802305262 time 705868079820 state 0x2 ref 0 00000100:00100000:0.0:1789760724.621681:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760724.621856:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195264 00000100:00100000:1.0:1789760724.621866:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195264 00000100:00100000:1.0:1789760724.621928:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705802305262 time 705868079820 state 0x0 ref 1 00000100:00100000:1.0:1789760724.621946:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705868340236 time 705868079820 state 0x1 ref 1 00000100:00100000:1.0:1789760724.621975:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705868340236 time 705937277575 state 0x0 ref 1 00000100:00100000:1.0:1789760724.621987:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507195264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760724.622034:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18124800, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:1.0:1789760724.622043:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.622059:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760724.622064:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.622070:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.622085:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.704526:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760724.704550:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538266496:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760724.704622:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760724.704713:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266496 00000100:00100000:2.0:1789760724.704721:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266496 00000100:00100000:2.0:1789760724.704769:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789760724.704781:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538266496:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760724.705093:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760724.705143:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538266496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 364us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760724.705150:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789760724.705194:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538266496:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760724.752910:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971745 is committed 00002000:00010000:1.0:1789760724.753126:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18124800 -> 18128896 00010000:00010000:1.0:1789760724.753141:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753155:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753164:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753171:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760724.753177:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753186:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753207:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.753214:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b753a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.753307:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507195264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 131317us (131631us total) trans 4294971745 rc 0/0 00000100:00100000:1.0:1789760724.753320:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4455 00000100:00100000:1.0:1789760724.753329:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705868340236 time 705937277575 state 0x2 ref 0 00000100:00100000:0.0:1789760724.759671:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.759834:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195392 00000100:00100000:2.0:1789760724.759845:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195392 00000100:00100000:2.0:1789760724.759922:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705868340236 time 705937277575 state 0x0 ref 1 00000100:00100000:2.0:1789760724.759939:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705937537991 time 705937277575 state 0x1 ref 1 00000100:00100000:0.0:1789760724.760114:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705937537991 time 706075414259 state 0x0 ref 1 00000100:00100000:0.0:1789760724.760130:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760724.760179:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18128896, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:0.0:1789760724.760189:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.760206:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760724.760211:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.760217:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.760230:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760724.809133:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18128896 -> 18132992 00010000:00010000:3.0:1789760724.809165:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809178:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809187:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809194:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.809199:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809209:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809229:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.809236:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7541 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.809338:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507195392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49207us (49673us total) trans 4294971746 rc 0/0 00000100:00100000:3.0:1789760724.809349:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4456 00000100:00100000:3.0:1789760724.809359:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705937537991 time 706075414259 state 0x2 ref 0 00000001:00080000:2.0:1789760724.810584:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971746 is committed 00000100:00100000:3.0:1789760724.811837:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.811922:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195520 00000100:00100000:2.0:1789760724.811930:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195520 00000100:00100000:2.0:1789760724.811983:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 705937537991 time 706075414259 state 0x0 ref 1 00000100:00100000:2.0:1789760724.811998:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706075674675 time 706075414259 state 0x1 ref 1 00000100:00100000:2.0:1789760724.812027:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706075674675 time 706127330325 state 0x0 ref 1 00000100:00100000:2.0:1789760724.812040:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507195520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.812087:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18132992, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.812095:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.812110:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760724.812115:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.812121:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.812133:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.862640:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18132992 -> 18137088 00010000:00010000:1.0:1789760724.862660:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862672:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862681:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862688:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760724.862720:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862732:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862752:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.862759:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7548 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.862892:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507195520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50849us (51057us total) trans 4294971747 rc 0/0 00000100:00100000:1.0:1789760724.863098:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4457 00000100:00100000:1.0:1789760724.863107:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706075674675 time 706127330325 state 0x2 ref 0 00000001:00080000:3.0:1789760724.864283:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971747 is committed 00000100:00100000:1.0:1789760724.867781:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760724.867903:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195648 00000100:00100000:3.0:1789760724.867924:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195648 00000100:00100000:3.0:1789760724.868001:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706075674675 time 706127330325 state 0x0 ref 1 00000100:00100000:3.0:1789760724.868021:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706127590741 time 706127330325 state 0x1 ref 1 00000100:00100000:3.0:1789760724.868052:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706127590741 time 706183355067 state 0x0 ref 1 00000100:00100000:3.0:1789760724.868067:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507195648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760724.868141:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18137088, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:3.0:1789760724.868152:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.868171:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760724.868176:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.868184:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.868199:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.891856:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760724.912564:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971748 is committed 00002000:00010000:3.0:1789760724.912762:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18137088 -> 18141184 00010000:00010000:3.0:1789760724.912783:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912798:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912808:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912816:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760724.912822:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912832:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912857:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.912865:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b754f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.913059:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507195648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44987us (45289us total) trans 4294971748 rc 0/0 00000100:00100000:3.0:1789760724.913070:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4458 00000100:00100000:3.0:1789760724.913081:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706127590741 time 706183355067 state 0x2 ref 0 00000100:00100000:2.0:1789760724.916601:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760724.916782:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195776 00000100:00100000:3.0:1789760724.916796:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195776 00000100:00100000:3.0:1789760724.916870:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706127590741 time 706183355067 state 0x0 ref 1 00000100:00100000:3.0:1789760724.916890:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706183615483 time 706183355067 state 0x1 ref 1 00000100:00100000:3.0:1789760724.916972:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706183615483 time 706232274207 state 0x0 ref 1 00000100:00100000:3.0:1789760724.916987:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760724.917051:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18141184, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:3.0:1789760724.917064:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.917085:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760724.917092:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.917100:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.917118:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.958365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971749 is committed 00002000:00010000:1.0:1789760724.958507:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18141184 -> 18145280 00010000:00010000:1.0:1789760724.958528:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958544:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958554:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958562:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760724.958568:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958579:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958605:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958614:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7556 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.958894:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41901us (42299us total) trans 4294971749 rc 0/0 00000100:00100000:1.0:1789760724.958907:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4459 00000100:00100000:1.0:1789760724.958929:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706183615483 time 706232274207 state 0x2 ref 0 00000100:00100000:1.0:1789760724.962150:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760724.962325:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507195904 00000100:00100000:2.0:1789760724.962339:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507195904 00000100:00100000:2.0:1789760724.962413:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706183615483 time 706232274207 state 0x0 ref 1 00000100:00100000:2.0:1789760724.962434:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706232534623 time 706232274207 state 0x1 ref 1 00000100:00100000:2.0:1789760724.962571:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706232534623 time 706277872153 state 0x0 ref 1 00000100:00100000:2.0:1789760724.962587:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760724.962651:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18145280, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:2.0:1789760724.962664:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.962726:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760724.962732:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.962741:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.962758:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.995150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971750 is committed 00002000:00010000:2.0:1789760724.995294:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18145280 -> 18149376 00010000:00010000:2.0:1789760724.995314:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995329:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995339:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995347:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760724.995353:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995363:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995392:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.995400:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b755d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18145280->18149375] (req 18145280->18149375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.996448:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507195904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33852us (34301us total) trans 4294971750 rc 0/0 00000100:00100000:2.0:1789760724.996462:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4460 00000100:00100000:2.0:1789760724.996547:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706232534623 time 706277872153 state 0x2 ref 0 00000100:00100000:3.0:1789760724.999512:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.000402:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196032 00000100:00100000:1.0:1789760725.000416:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196032 00000100:00100000:1.0:1789760725.000490:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706232534623 time 706277872153 state 0x0 ref 1 00000100:00100000:1.0:1789760725.000509:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706278132569 time 706277872153 state 0x1 ref 1 00000100:00100000:1.0:1789760725.000548:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706278132569 time 706315851181 state 0x0 ref 1 00000100:00100000:1.0:1789760725.000562:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.000630:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18149376, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9007 00010000:00010000:1.0:1789760725.000642:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.000662:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.000668:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.000677:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.000726:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.091114:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971751 is committed 00002000:00010000:1.0:1789760725.091463:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18149376 -> 18153472 00002000:00010000:1.0:1789760725.091469:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760723 -> 1789760724 00002000:00010000:1.0:1789760725.091472:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760723 -> 1789760724 00010000:00010000:1.0:1789760725.091490:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091505:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091515:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091522:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.091528:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091539:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091565:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.091578:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7564 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18149376->18153471] (req 18149376->18153471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.091751:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91184us (92246us total) trans 4294971751 rc 0/0 00000100:00100000:1.0:1789760725.091766:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4461 00000100:00100000:1.0:1789760725.091779:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706278132569 time 706315851181 state 0x2 ref 0 00000100:00100000:3.0:1789760725.094380:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.094473:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196160 00000100:00100000:1.0:1789760725.094486:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196160 00000100:00100000:1.0:1789760725.094578:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706278132569 time 706315851181 state 0x0 ref 1 00000100:00100000:1.0:1789760725.094598:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706316111597 time 706315851181 state 0x1 ref 1 00000100:00100000:1.0:1789760725.094632:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706316111597 time 706409934511 state 0x0 ref 1 00000100:00100000:1.0:1789760725.094646:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.094746:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18153472, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.094757:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.094777:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.094783:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.094790:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.094813:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760725.095581:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760725.163849:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18153472 -> 18157568 00010000:00010000:3.0:1789760725.163877:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.163894:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.163906:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.164038:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760725.164045:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.164056:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.164081:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.164208:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971752 is committed 00010000:00010000:3.0:1789760725.164237:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b756b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.164369:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69718us (70007us total) trans 4294971752 rc 0/0 00000100:00100000:3.0:1789760725.164383:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4462 00000100:00100000:3.0:1789760725.164394:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706316111597 time 706409934511 state 0x2 ref 0 00000100:00100000:3.0:1789760725.170274:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.173939:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196288 00000100:00100000:2.0:1789760725.173953:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196288 00000100:00100000:3.0:1789760725.174121:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706316111597 time 706409934511 state 0x0 ref 1 00000100:00100000:3.0:1789760725.174136:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706410194927 time 706409934511 state 0x1 ref 1 00000100:00100000:3.0:1789760725.174171:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706410194927 time 706489474126 state 0x0 ref 1 00000100:00100000:3.0:1789760725.174192:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760725.174242:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18157568, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:3.0:1789760725.174251:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.174265:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760725.174270:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.174276:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.174296:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.216036:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266624:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760725.216267:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.216375:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266624 00000100:00100000:0.0:1789760725.216383:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266624 00000100:00100000:0.0:1789760725.216440:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:0.0:1789760725.216449:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538266624:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760725.216486:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760725.216539:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538266624:12345-0@lo:41:osp-pre-0-1.0 Request processed in 91us (280us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.216546:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:0.0:1789760725.216608:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266624:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760725.223310:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971753 is committed 00002000:00010000:3.0:1789760725.223574:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18157568 -> 18161664 00010000:00010000:3.0:1789760725.223594:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223610:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223620:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223639:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760725.223645:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223655:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223681:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.223690:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7572 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.223833:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49641us (53838us total) trans 4294971753 rc 0/0 00000100:00100000:3.0:1789760725.223846:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4463 00000100:00100000:3.0:1789760725.223859:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706410194927 time 706489474126 state 0x2 ref 0 00000100:00100000:1.0:1789760725.227989:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760725.228122:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196416 00000100:00100000:0.0:1789760725.228137:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196416 00000100:00100000:0.0:1789760725.228214:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706410194927 time 706489474126 state 0x0 ref 1 00000100:00100000:0.0:1789760725.228235:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706489734542 time 706489474126 state 0x1 ref 1 00000100:00100000:0.0:1789760725.228267:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706489734542 time 706543570699 state 0x0 ref 1 00000100:00100000:0.0:1789760725.228283:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760725.228361:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18161664, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:0.0:1789760725.228373:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228470:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760725.228476:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.228484:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228502:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.266790:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971754 is committed 00002000:00010000:1.0:1789760725.266903:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18161664 -> 18165760 00010000:00010000:1.0:1789760725.266934:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267069:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267081:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267089:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.267094:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267105:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267130:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.267138:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7579 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.267308:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39010us (39314us total) trans 4294971754 rc 0/0 00000100:00100000:1.0:1789760725.267321:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4464 00000100:00100000:1.0:1789760725.267334:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706489734542 time 706543570699 state 0x2 ref 0 00000100:00100000:3.0:1789760725.271789:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.271878:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196544 00000100:00100000:1.0:1789760725.271892:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196544 00000100:00100000:1.0:1789760725.271989:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706489734542 time 706543570699 state 0x0 ref 1 00000100:00100000:1.0:1789760725.272018:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706543831115 time 706543570699 state 0x1 ref 1 00000100:00100000:1.0:1789760725.272058:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706543831115 time 706587360769 state 0x0 ref 1 00000100:00100000:1.0:1789760725.272073:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.272139:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18165760, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.272149:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.272170:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.272175:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.272183:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.272202:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.308214:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971755 is committed 00002000:00010000:0.0:1789760725.308559:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18165760 -> 18169856 00010000:00010000:0.0:1789760725.308580:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308596:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308612:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308621:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760725.308627:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308638:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308664:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.308673:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7580 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.308829:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36751us (37046us total) trans 4294971755 rc 0/0 00000100:00100000:0.0:1789760725.308840:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4465 00000100:00100000:0.0:1789760725.308852:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706543831115 time 706587360769 state 0x2 ref 0 00000100:00100000:0.0:1789760725.315752:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760725.315938:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196672 00000100:00100000:3.0:1789760725.315950:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196672 00000100:00100000:3.0:1789760725.316025:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706543831115 time 706587360769 state 0x0 ref 1 00000100:00100000:3.0:1789760725.316045:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706587621185 time 706587360769 state 0x1 ref 1 00000100:00100000:3.0:1789760725.316087:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706587621185 time 706631389855 state 0x0 ref 1 00000100:00100000:3.0:1789760725.316102:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760725.316169:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18169856, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:3.0:1789760725.316180:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.316200:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760725.316205:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.316213:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.316236:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.354539:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971756 is committed 00002000:00010000:1.0:1789760725.354737:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18169856 -> 18173952 00010000:00010000:1.0:1789760725.354757:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354772:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354782:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354789:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.354795:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354805:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354829:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.354836:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7587 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.354977:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507196672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38870us (39234us total) trans 4294971756 rc 0/0 00000100:00100000:1.0:1789760725.354991:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4466 00000100:00100000:1.0:1789760725.355012:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706587621185 time 706631389855 state 0x2 ref 0 00000100:00100000:0.0:1789760725.360355:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.360764:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196800 00000100:00100000:1.0:1789760725.360779:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196800 00000100:00100000:1.0:1789760725.360860:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706587621185 time 706631389855 state 0x0 ref 1 00000100:00100000:1.0:1789760725.360882:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706631650271 time 706631389855 state 0x1 ref 1 00000100:00100000:1.0:1789760725.360933:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706631650271 time 706676234546 state 0x0 ref 1 00000100:00100000:1.0:1789760725.360948:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.361014:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18173952, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.361025:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.361043:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.361048:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.361055:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.361072:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.427769:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971757 is committed 00002000:00010000:0.0:1789760725.427785:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18173952 -> 18178048 00010000:00010000:0.0:1789760725.427808:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427824:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427835:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427842:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760725.427848:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427857:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427883:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.427892:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b758e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.428041:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67088us (67694us total) trans 4294971757 rc 0/0 00000100:00100000:0.0:1789760725.428054:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4467 00000100:00100000:0.0:1789760725.428066:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706631650271 time 706676234546 state 0x2 ref 0 00000100:00100000:3.0:1789760725.432353:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760725.432435:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507196928 00000100:00100000:0.0:1789760725.432450:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507196928 00000100:00100000:0.0:1789760725.432524:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706631650271 time 706676234546 state 0x0 ref 1 00000100:00100000:0.0:1789760725.432543:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706676494962 time 706676234546 state 0x1 ref 1 00000100:00100000:0.0:1789760725.432590:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706676494962 time 706747892770 state 0x0 ref 1 00000100:00100000:0.0:1789760725.432605:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760725.432682:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18178048, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:0.0:1789760725.432727:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.432748:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760725.432754:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.432762:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.432782:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.460850:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971758 is committed 00002000:00010000:2.0:1789760725.461098:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18178048 -> 18182144 00010000:00010000:2.0:1789760725.461118:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461204:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461216:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461230:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760725.461237:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461248:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461274:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.461283:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7595 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.461453:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507196928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28842us (29107us total) trans 4294971758 rc 0/0 00000100:00100000:2.0:1789760725.461467:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4468 00000100:00100000:2.0:1789760725.461480:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706676494962 time 706747892770 state 0x2 ref 0 00000100:00100000:3.0:1789760725.466824:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.466951:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197056 00000100:00100000:1.0:1789760725.466974:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197056 00000100:00100000:1.0:1789760725.467050:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706676494962 time 706747892770 state 0x0 ref 1 00000100:00100000:1.0:1789760725.467070:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706748153186 time 706747892770 state 0x1 ref 1 00000100:00100000:1.0:1789760725.467105:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706748153186 time 706782407849 state 0x0 ref 1 00000100:00100000:1.0:1789760725.467120:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507197056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.467195:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18182144, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.467206:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.467227:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.467232:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.467240:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.467258:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.526899:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971759 is committed 00002000:00010000:1.0:1789760725.527048:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18182144 -> 18186240 00010000:00010000:1.0:1789760725.527063:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527134:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527145:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527153:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.527158:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527167:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527184:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.527192:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b759c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.527298:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507197056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60178us (60484us total) trans 4294971759 rc 0/0 00000100:00100000:1.0:1789760725.527307:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4469 00000100:00100000:1.0:1789760725.527316:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706748153186 time 706782407849 state 0x2 ref 0 00000100:00100000:0.0:1789760725.537553:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.537773:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197184 00000100:00100000:1.0:1789760725.537784:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197184 00000100:00100000:1.0:1789760725.537841:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706748153186 time 706782407849 state 0x0 ref 1 00000100:00100000:1.0:1789760725.537857:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706782668265 time 706782407849 state 0x1 ref 1 00000100:00100000:1.0:1789760725.537892:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706782668265 time 706853194473 state 0x0 ref 1 00000100:00100000:1.0:1789760725.537918:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507197184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.537984:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18186240, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.537993:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.538010:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.538014:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.538022:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.538036:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.595368:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971760 is committed 00002000:00010000:1.0:1789760725.595531:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18186240 -> 18190336 00010000:00010000:1.0:1789760725.595548:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595560:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595569:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595577:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.595582:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595594:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595614:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.595622:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.595761:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507197184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57850us (58213us total) trans 4294971760 rc 0/0 00000100:00100000:1.0:1789760725.595771:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4470 00000100:00100000:1.0:1789760725.595783:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706782668265 time 706853194473 state 0x2 ref 0 00000100:00100000:3.0:1789760725.600327:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.600418:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197312 00000100:00100000:2.0:1789760725.600427:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197312 00000100:00100000:2.0:1789760725.600485:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706782668265 time 706853194473 state 0x0 ref 1 00000100:00100000:2.0:1789760725.600501:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706853454889 time 706853194473 state 0x1 ref 1 00000100:00100000:2.0:1789760725.600535:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706853454889 time 706915838436 state 0x0 ref 1 00000100:00100000:2.0:1789760725.600551:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.600598:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18190336, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.600606:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.600621:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760725.600626:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.600633:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.600646:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.651733:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971761 is committed 00002000:00010000:1.0:1789760725.651835:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18190336 -> 18194432 00010000:00010000:1.0:1789760725.651852:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.651865:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.651874:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.651882:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.651887:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.651897:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.652101:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.652110:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b75aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18190336->18194431] (req 18190336->18194431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.652246:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51691us (51923us total) trans 4294971761 rc 0/0 00000100:00100000:1.0:1789760725.652258:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4471 00000100:00100000:1.0:1789760725.652267:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706853454889 time 706915838436 state 0x2 ref 0 00000100:00100000:0.0:1789760725.656332:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760725.656411:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197440 00000100:00100000:1.0:1789760725.656420:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197440 00000100:00100000:1.0:1789760725.656471:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706853454889 time 706915838436 state 0x0 ref 1 00000100:00100000:1.0:1789760725.656485:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706916098852 time 706915838436 state 0x1 ref 1 00000100:00100000:1.0:1789760725.656512:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706916098852 time 706971815873 state 0x0 ref 1 00000100:00100000:1.0:1789760725.656540:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760725.656598:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18194432, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:1.0:1789760725.656609:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.656625:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760725.656629:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.656636:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.656650:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.688970:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971762 is committed 00002000:00010000:1.0:1789760725.689035:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18194432 -> 18198528 00010000:00010000:1.0:1789760725.689049:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689061:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689070:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689077:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.689082:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689091:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689114:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.689121:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b75b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.689206:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32679us (32879us total) trans 4294971762 rc 0/0 00000100:00100000:1.0:1789760725.689215:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4472 00000100:00100000:1.0:1789760725.689224:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706916098852 time 706971815873 state 0x2 ref 0 00000100:00100000:1.0:1789760725.693927:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.694099:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197568 00000100:00100000:2.0:1789760725.694109:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197568 00000100:00100000:2.0:1789760725.694162:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706916098852 time 706971815873 state 0x0 ref 1 00000100:00100000:2.0:1789760725.694177:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706972076289 time 706971815873 state 0x1 ref 1 00000100:00100000:2.0:1789760725.694210:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706972076289 time 707009513935 state 0x0 ref 1 00000100:00100000:2.0:1789760725.694223:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.694270:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18198528, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.694279:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.694300:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.694304:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.694311:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.694325:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.725454:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971763 is committed 00002000:00010000:0.0:1789760725.725553:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18198528 -> 18202624 00010000:00010000:0.0:1789760725.725567:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725581:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725591:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725599:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760725.725604:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725613:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725639:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.725647:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b75b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.725766:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31540us (31858us total) trans 4294971763 rc 0/0 00000100:00100000:0.0:1789760725.725775:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4473 00000100:00100000:0.0:1789760725.725784:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706972076289 time 707009513935 state 0x2 ref 0 00000100:00100000:1.0:1789760725.729366:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.729453:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197696 00000100:00100000:2.0:1789760725.729462:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197696 00000100:00100000:2.0:1789760725.729516:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 706972076289 time 707009513935 state 0x0 ref 1 00000100:00100000:2.0:1789760725.729530:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707009774351 time 707009513935 state 0x1 ref 1 00000100:00100000:2.0:1789760725.729562:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707009774351 time 707044865425 state 0x0 ref 1 00000100:00100000:2.0:1789760725.729574:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.729622:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18202624, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.729632:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.729649:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.729653:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.729660:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.729680:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.751316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971764 is committed 00002000:00010000:2.0:1789760725.751434:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18202624 -> 18206720 00010000:00010000:2.0:1789760725.751448:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751462:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751471:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751478:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760725.751483:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751492:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751510:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.751517:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.751606:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22029us (22255us total) trans 4294971764 rc 0/0 00000100:00100000:2.0:1789760725.751616:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4474 00000100:00100000:2.0:1789760725.751625:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707009774351 time 707044865425 state 0x2 ref 0 00000100:00100000:1.0:1789760725.756504:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.756593:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197824 00000100:00100000:2.0:1789760725.756604:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197824 00000100:00100000:2.0:1789760725.756658:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707009774351 time 707044865425 state 0x0 ref 1 00000100:00100000:2.0:1789760725.756674:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707045125841 time 707044865425 state 0x1 ref 1 00000100:00100000:2.0:1789760725.756729:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707045125841 time 707072031750 state 0x0 ref 1 00000100:00100000:2.0:1789760725.756741:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.756787:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18206720, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.756800:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.756816:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.756821:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.756827:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.756840:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.778634:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971765 is committed 00002000:00010000:1.0:1789760725.778745:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18206720 -> 18210816 00010000:00010000:1.0:1789760725.778761:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778773:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778783:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778795:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760725.778801:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778810:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778828:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.778836:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b75c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.778935:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507197824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22192us (22436us total) trans 4294971765 rc 0/0 00000100:00100000:1.0:1789760725.778951:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4475 00000100:00100000:1.0:1789760725.778962:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707045125841 time 707072031750 state 0x2 ref 0 00000100:00100000:1.0:1789760725.782771:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760725.782853:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507197952 00000100:00100000:0.0:1789760725.782862:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507197952 00000100:00100000:0.0:1789760725.782928:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707045125841 time 707072031750 state 0x0 ref 1 00000100:00100000:0.0:1789760725.783004:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707072292166 time 707072031750 state 0x1 ref 1 00000100:00100000:0.0:1789760725.783035:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707072292166 time 707098338522 state 0x0 ref 1 00000100:00100000:0.0:1789760725.783048:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507197952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760725.783097:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18210816, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:0.0:1789760725.783104:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.783121:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760725.783126:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.783132:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.783146:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.811511:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971766 is committed 00002000:00010000:0.0:1789760725.811608:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18210816 -> 18214912 00010000:00010000:0.0:1789760725.811621:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811633:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811642:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811653:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760725.811658:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811666:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811684:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.811706:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.811798:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507197952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28747us (29031us total) trans 4294971766 rc 0/0 00000100:00100000:0.0:1789760725.811807:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4476 00000100:00100000:0.0:1789760725.811816:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707072292166 time 707098338522 state 0x2 ref 0 00000100:00100000:0.0:1789760725.816824:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.816978:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198080 00000100:00100000:2.0:1789760725.816987:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198080 00000100:00100000:2.0:1789760725.817270:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707072292166 time 707098338522 state 0x0 ref 1 00000100:00100000:2.0:1789760725.817284:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707098598938 time 707098338522 state 0x1 ref 1 00000100:00100000:2.0:1789760725.817316:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707098598938 time 707132619552 state 0x0 ref 1 00000100:00100000:2.0:1789760725.817329:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507198080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.817378:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18214912, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.817387:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.817403:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.817407:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.817414:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.817428:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.843267:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971767 is committed 00002000:00010000:3.0:1789760725.843363:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18214912 -> 18219008 00010000:00010000:3.0:1789760725.843378:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843495:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843504:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843511:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760725.843516:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843524:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843546:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.843553:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b75d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.843650:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507198080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26319us (26831us total) trans 4294971767 rc 0/0 00000100:00100000:3.0:1789760725.843661:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4477 00000100:00100000:3.0:1789760725.843669:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707098598938 time 707132619552 state 0x2 ref 0 00000100:00100000:1.0:1789760725.847294:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.847381:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198208 00000100:00100000:2.0:1789760725.847453:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198208 00000100:00100000:2.0:1789760725.847511:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707098598938 time 707132619552 state 0x0 ref 1 00000100:00100000:2.0:1789760725.847527:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707132879968 time 707132619552 state 0x1 ref 1 00000100:00100000:2.0:1789760725.847591:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707132879968 time 707162893482 state 0x0 ref 1 00000100:00100000:2.0:1789760725.847603:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507198208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.847651:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18219008, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.847661:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.847678:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.847682:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.847688:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.847735:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.881284:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18219008 -> 18223104 00010000:00010000:0.0:1789760725.881304:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881316:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881331:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881337:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760725.881341:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881350:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881368:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.881375:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.881474:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507198208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33868us (34184us total) trans 4294971768 rc 0/0 00000100:00100000:0.0:1789760725.881484:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4478 00000100:00100000:0.0:1789760725.881493:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707132879968 time 707162893482 state 0x2 ref 0 00000001:00080000:1.0:1789760725.881800:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971768 is committed 00000100:00100000:1.0:1789760725.889399:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760725.889594:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198336 00000100:00100000:2.0:1789760725.889603:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198336 00000100:00100000:2.0:1789760725.889662:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707132879968 time 707162893482 state 0x0 ref 1 00000100:00100000:2.0:1789760725.889677:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707163153898 time 707162893482 state 0x1 ref 1 00000100:00100000:2.0:1789760725.889731:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707163153898 time 707205034179 state 0x0 ref 1 00000100:00100000:2.0:1789760725.889745:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.889793:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18223104, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.889802:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.889818:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.889823:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.889830:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.889843:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.915809:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760725.917323:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971769 is committed 00002000:00010000:2.0:1789760725.917486:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18223104 -> 18227200 00010000:00010000:2.0:1789760725.917501:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917513:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917523:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917530:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760725.917535:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917545:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917571:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.917579:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.917741:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27994us (28348us total) trans 4294971769 rc 0/0 00000100:00100000:2.0:1789760725.917751:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4479 00000100:00100000:2.0:1789760725.917761:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707163153898 time 707205034179 state 0x2 ref 0 00000100:00100000:2.0:1789760725.922583:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760725.922679:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198464 00000100:00100000:3.0:1789760725.922689:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198464 00000100:00100000:3.0:1789760725.922863:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707163153898 time 707205034179 state 0x0 ref 1 00000100:00100000:3.0:1789760725.922878:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707205294595 time 707205034179 state 0x1 ref 1 00000100:00100000:2.0:1789760725.923123:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707205294595 time 707238423563 state 0x0 ref 1 00000100:00100000:2.0:1789760725.923140:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760725.923278:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18227200, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760725.923287:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.923302:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760725.923306:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.923312:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.923326:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.960229:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971770 is committed 00002000:00010000:3.0:1789760725.960335:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18227200 -> 18231296 00010000:00010000:3.0:1789760725.960350:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960364:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960373:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960380:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760725.960385:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960393:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960413:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.960422:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b75e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.960550:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37410us (37972us total) trans 4294971770 rc 0/0 00000100:00100000:3.0:1789760725.960561:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4480 00000100:00100000:3.0:1789760725.960570:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707205294595 time 707238423563 state 0x2 ref 0 00000100:00100000:1.0:1789760725.963284:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760725.963875:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198592 00000100:00100000:0.0:1789760725.963886:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198592 00000100:00100000:0.0:1789760725.963958:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707205294595 time 707238423563 state 0x0 ref 1 00000100:00100000:0.0:1789760725.963975:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707238683979 time 707238423563 state 0x1 ref 1 00000100:00100000:0.0:1789760725.964043:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707238683979 time 707279346090 state 0x0 ref 1 00000100:00100000:0.0:1789760725.964063:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760725.964114:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18231296, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:0.0:1789760725.964123:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.964137:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760725.964141:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.964157:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.964171:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760726.027369:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18231296 -> 18235392 00010000:00010000:2.0:1789760726.027389:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027402:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027420:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027427:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760726.027432:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027441:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027461:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.027468:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b75f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.027490:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971771 is committed 00000100:00100000:2.0:1789760726.027597:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63538us (64318us total) trans 4294971771 rc 0/0 00000100:00100000:2.0:1789760726.027610:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4481 00000100:00100000:2.0:1789760726.027619:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707238683979 time 707279346090 state 0x2 ref 0 00000100:00100000:1.0:1789760726.035227:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.035409:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198720 00000100:00100000:2.0:1789760726.035418:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198720 00000100:00100000:2.0:1789760726.035473:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707238683979 time 707279346090 state 0x0 ref 1 00000100:00100000:2.0:1789760726.035488:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707279606506 time 707279346090 state 0x1 ref 1 00000100:00100000:2.0:1789760726.038795:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707279606506 time 707354086304 state 0x0 ref 1 00000100:00100000:2.0:1789760726.038826:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507198720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.038881:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18235392, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9007 00010000:00010000:2.0:1789760726.038890:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.038916:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760726.038922:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.038928:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.039310:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.083912:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971772 is committed 00002000:00010000:2.0:1789760726.084119:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18235392 -> 18239488 00002000:00010000:2.0:1789760726.084125:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760724 -> 1789760725 00002000:00010000:2.0:1789760726.084128:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760724 -> 1789760725 00010000:00010000:2.0:1789760726.084140:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084155:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084164:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084173:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760726.084178:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084187:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084206:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.084213:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b75f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.084343:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507198720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45527us (49120us total) trans 4294971772 rc 0/0 00000100:00100000:2.0:1789760726.084354:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4482 00000100:00100000:2.0:1789760726.084363:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707279606506 time 707354086304 state 0x2 ref 0 00000100:00100000:0.0:1789760726.092489:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760726.092567:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198848 00000100:00100000:3.0:1789760726.092577:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198848 00000100:00100000:3.0:1789760726.092631:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707279606506 time 707354086304 state 0x0 ref 1 00000100:00100000:3.0:1789760726.092646:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707354346720 time 707354086304 state 0x1 ref 1 00000100:00100000:3.0:1789760726.092671:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707354346720 time 707407975146 state 0x0 ref 1 00000100:00100000:3.0:1789760726.092684:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760726.092757:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18239488, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760726.092766:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.092782:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760726.092786:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.092793:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.092807:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.137845:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971773 is committed 00002000:00010000:0.0:1789760726.138153:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18239488 -> 18243584 00010000:00010000:0.0:1789760726.138168:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138179:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138187:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138198:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760726.138203:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138211:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138233:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.138241:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b75fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.138330:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507198848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45644us (45846us total) trans 4294971773 rc 0/0 00000100:00100000:0.0:1789760726.138341:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4483 00000100:00100000:0.0:1789760726.138350:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707354346720 time 707407975146 state 0x2 ref 0 00000100:00100000:1.0:1789760726.142564:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.144280:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507198976 00000100:00100000:2.0:1789760726.144292:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507198976 00000100:00100000:2.0:1789760726.144350:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707354346720 time 707407975146 state 0x0 ref 1 00000100:00100000:2.0:1789760726.144366:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707408235562 time 707407975146 state 0x1 ref 1 00000100:00100000:2.0:1789760726.144454:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707408235562 time 707459755929 state 0x0 ref 1 00000100:00100000:2.0:1789760726.144468:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507198976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.144516:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18243584, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:2.0:1789760726.144525:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.144540:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760726.144544:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.144551:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.144565:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.176298:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971774 is committed 00002000:00010000:1.0:1789760726.176956:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18243584 -> 18247680 00010000:00010000:1.0:1789760726.176971:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.176984:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.176993:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.177000:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760726.177004:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.177015:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.177034:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.177041:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7605 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.177140:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507198976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32670us (34581us total) trans 4294971774 rc 0/0 00000100:00100000:1.0:1789760726.177151:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4484 00000100:00100000:1.0:1789760726.177161:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707408235562 time 707459755929 state 0x2 ref 0 00000100:00100000:1.0:1789760726.192423:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.193454:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199104 00000100:00100000:2.0:1789760726.193466:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199104 00000100:00100000:3.0:1789760726.194176:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707408235562 time 707459755929 state 0x0 ref 1 00000100:00100000:3.0:1789760726.194193:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707460016345 time 707459755929 state 0x1 ref 1 00000100:00100000:3.0:1789760726.194273:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707460016345 time 707509574869 state 0x0 ref 1 00000100:00100000:3.0:1789760726.194287:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507199104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760726.194337:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18247680, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760726.194347:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.194375:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760726.194380:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.194387:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.194404:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.239757:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971775 is committed 00002000:00010000:0.0:1789760726.239852:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18247680 -> 18251776 00010000:00010000:0.0:1789760726.239866:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239879:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239895:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239912:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760726.239917:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239926:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239953:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.239960:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b760c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.240052:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507199104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45763us (47634us total) trans 4294971775 rc 0/0 00000100:00100000:0.0:1789760726.240062:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4485 00000100:00100000:0.0:1789760726.240072:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707460016345 time 707509574869 state 0x2 ref 0 00000100:00100000:0.0:1789760726.246062:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760726.246166:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199232 00000100:00100000:3.0:1789760726.246176:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199232 00000100:00100000:3.0:1789760726.246229:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707460016345 time 707509574869 state 0x0 ref 1 00000100:00100000:3.0:1789760726.246244:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707509835285 time 707509574869 state 0x1 ref 1 00000100:00100000:3.0:1789760726.246276:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707509835285 time 707561579548 state 0x0 ref 1 00000100:00100000:3.0:1789760726.246289:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507199232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760726.246548:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18251776, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760726.246558:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.246574:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760726.246579:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.246585:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.246601:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.282739:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971776 is committed 00002000:00010000:3.0:1789760726.283132:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18251776 -> 18255872 00010000:00010000:3.0:1789760726.283147:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283160:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283170:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283178:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.283182:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283192:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283212:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.283220:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7613 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.283446:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507199232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37154us (37386us total) trans 4294971776 rc 0/0 00000100:00100000:3.0:1789760726.283458:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4486 00000100:00100000:3.0:1789760726.283468:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707509835285 time 707561579548 state 0x2 ref 0 00000100:00100000:3.0:1789760726.294179:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760726.294672:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199360 00000100:00100000:1.0:1789760726.294682:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199360 00000100:00100000:1.0:1789760726.294779:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707509835285 time 707561579548 state 0x0 ref 1 00000100:00100000:1.0:1789760726.294795:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707561839964 time 707561579548 state 0x1 ref 1 00000100:00100000:1.0:1789760726.294893:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707561839964 time 707610194755 state 0x0 ref 1 00000100:00100000:1.0:1789760726.294919:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507199360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760726.294978:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18255872, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:1.0:1789760726.294987:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.295004:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760726.295009:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.295015:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.295031:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.338425:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18255872 -> 18259968 00010000:00010000:1.0:1789760726.338442:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338460:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338470:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338483:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760726.338488:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338498:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338516:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338523:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b761a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.338611:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507199360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43693us (44439us total) trans 4294971777 rc 0/0 00000100:00100000:1.0:1789760726.338620:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4487 00000100:00100000:1.0:1789760726.338631:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707561839964 time 707610194755 state 0x2 ref 0 00000001:00080000:2.0:1789760726.342015:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971777 is committed 00000100:00100000:1.0:1789760726.345060:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760726.345124:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199488 00000100:00100000:0.0:1789760726.345134:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199488 00000100:00100000:0.0:1789760726.345186:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707561839964 time 707610194755 state 0x0 ref 1 00000100:00100000:0.0:1789760726.345200:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707610455171 time 707610194755 state 0x1 ref 1 00000100:00100000:0.0:1789760726.345223:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707610455171 time 707660527406 state 0x0 ref 1 00000100:00100000:0.0:1789760726.345231:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507199488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760726.345274:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18259968, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:0.0:1789760726.345283:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.345299:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760726.345304:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.345310:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.345324:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.382308:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18259968 -> 18264064 00010000:00010000:0.0:1789760726.382330:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382343:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382353:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382366:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760726.382372:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382383:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382405:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.382414:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7621 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.382507:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507199488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37269us (37448us total) trans 4294971778 rc 0/0 00000100:00100000:0.0:1789760726.382519:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4488 00000100:00100000:0.0:1789760726.382530:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707610455171 time 707660527406 state 0x2 ref 0 00000001:00080000:2.0:1789760726.382749:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971778 is committed 00000100:00100000:3.0:1789760726.389207:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760726.389319:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199616 00000100:00100000:0.0:1789760726.389329:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199616 00000100:00100000:0.0:1789760726.389386:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707610455171 time 707660527406 state 0x0 ref 1 00000100:00100000:0.0:1789760726.389401:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707660787822 time 707660527406 state 0x1 ref 1 00000100:00100000:0.0:1789760726.389432:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707660787822 time 707704735537 state 0x0 ref 1 00000100:00100000:0.0:1789760726.389448:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507199616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760726.389499:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18264064, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:0.0:1789760726.389508:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389522:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760726.389527:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.389534:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389547:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.428297:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971779 is committed 00002000:00010000:2.0:1789760726.428373:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18264064 -> 18268160 00010000:00010000:2.0:1789760726.428389:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428402:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428478:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428485:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760726.428490:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428498:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428517:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.428525:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7628 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.428647:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507199616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39196us (39552us total) trans 4294971779 rc 0/0 00000100:00100000:2.0:1789760726.428657:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4489 00000100:00100000:2.0:1789760726.428667:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707660787822 time 707704735537 state 0x2 ref 0 00000100:00100000:3.0:1789760726.440635:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760726.440926:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199744 00000100:00100000:1.0:1789760726.441506:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199744 00000100:00100000:1.0:1789760726.441596:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707660787822 time 707704735537 state 0x0 ref 1 00000100:00100000:1.0:1789760726.441617:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707704995953 time 707704735537 state 0x1 ref 1 00000100:00100000:1.0:1789760726.441658:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707704995953 time 707756960290 state 0x0 ref 1 00000100:00100000:1.0:1789760726.442515:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507199744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760726.442597:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18268160, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:1.0:1789760726.442608:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.442629:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760726.442635:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.442644:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.442661:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.504350:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971780 is committed 00002000:00010000:0.0:1789760726.504463:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18268160 -> 18272256 00010000:00010000:0.0:1789760726.504546:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504564:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504575:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504584:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760726.504590:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504601:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504628:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.504637:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b762f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.504948:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507199744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63259us (64311us total) trans 4294971780 rc 0/0 00000100:00100000:0.0:1789760726.504962:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4490 00000100:00100000:0.0:1789760726.504975:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707704995953 time 707756960290 state 0x2 ref 0 00000100:00100000:0.0:1789760726.511399:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.511576:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507199872 00000100:00100000:2.0:1789760726.511589:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507199872 00000100:00100000:2.0:1789760726.511662:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707704995953 time 707756960290 state 0x0 ref 1 00000100:00100000:2.0:1789760726.511708:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707757220706 time 707756960290 state 0x1 ref 1 00000100:00100000:2.0:1789760726.511743:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707757220706 time 707827045629 state 0x0 ref 1 00000100:00100000:2.0:1789760726.511757:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507199872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.511821:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18272256, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:2.0:1789760726.511831:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.511850:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760726.511855:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.511863:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.511881:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.562067:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971781 is committed 00002000:00010000:3.0:1789760726.562204:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18272256 -> 18276352 00010000:00010000:3.0:1789760726.562224:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562239:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562250:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562258:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.562263:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562274:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562305:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.562314:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b7636 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.562447:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507199872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50684us (51054us total) trans 4294971781 rc 0/0 00000100:00100000:3.0:1789760726.562459:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4491 00000100:00100000:3.0:1789760726.562545:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707757220706 time 707827045629 state 0x2 ref 0 00000100:00100000:1.0:1789760726.567388:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.567519:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200000 00000100:00100000:2.0:1789760726.567533:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200000 00000100:00100000:2.0:1789760726.567618:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707757220706 time 707827045629 state 0x0 ref 1 00000100:00100000:2.0:1789760726.567643:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707827306045 time 707827045629 state 0x1 ref 1 00000100:00100000:2.0:1789760726.567689:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707827306045 time 707882991144 state 0x0 ref 1 00000100:00100000:2.0:1789760726.567749:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507200000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.567821:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18276352, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:2.0:1789760726.567831:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.567855:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760726.567860:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.567869:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.567885:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.635734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971782 is committed 00002000:00010000:3.0:1789760726.636006:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18276352 -> 18280448 00010000:00010000:3.0:1789760726.636026:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636040:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636049:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636057:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.636062:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636075:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636095:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.636103:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b763d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.636237:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507200000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68486us (68863us total) trans 4294971782 rc 0/0 00000100:00100000:3.0:1789760726.636248:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4492 00000100:00100000:3.0:1789760726.636258:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707827306045 time 707882991144 state 0x2 ref 0 00000100:00100000:2.0:1789760726.642769:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760726.642861:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200128 00000100:00100000:0.0:1789760726.642872:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200128 00000100:00100000:0.0:1789760726.642937:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707827306045 time 707882991144 state 0x0 ref 1 00000100:00100000:0.0:1789760726.642954:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707883251560 time 707882991144 state 0x1 ref 1 00000100:00100000:0.0:1789760726.642985:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707883251560 time 707958287953 state 0x0 ref 1 00000100:00100000:0.0:1789760726.642997:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507200128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760726.643052:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18280448, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:0.0:1789760726.643061:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.643077:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760726.643082:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.643089:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.643104:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.731072:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18280448 -> 18284544 00010000:00010000:1.0:1789760726.731092:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731105:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731113:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731120:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760726.731125:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731134:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731156:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.731164:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7644 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18280448->18284543] (req 18280448->18284543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.731267:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507200128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 88266us (88505us total) trans 4294971783 rc 0/0 00000100:00100000:1.0:1789760726.731277:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4493 00000100:00100000:1.0:1789760726.731287:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707883251560 time 707958287953 state 0x2 ref 0 00000001:00080000:1.0:1789760726.731340:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971783 is committed 00000100:00100000:1.0:1789760726.738457:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760726.738547:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200256 00000100:00100000:3.0:1789760726.738557:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200256 00000100:00100000:3.0:1789760726.738612:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707883251560 time 707958287953 state 0x0 ref 1 00000100:00100000:3.0:1789760726.738628:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707958548369 time 707958287953 state 0x1 ref 1 00000100:00100000:3.0:1789760726.738661:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707958548369 time 708053963935 state 0x0 ref 1 00000100:00100000:3.0:1789760726.738674:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507200256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760726.738754:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18284544, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760726.738763:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.738779:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760726.738783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.738788:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.738803:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.810608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971784 is committed 00002000:00010000:3.0:1789760726.810811:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18284544 -> 18288640 00010000:00010000:3.0:1789760726.810825:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810836:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810844:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810850:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.810855:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810864:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810882:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.810890:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b764b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.810995:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507200256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72320us (72544us total) trans 4294971784 rc 0/0 00000100:00100000:3.0:1789760726.811004:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4494 00000100:00100000:3.0:1789760726.811013:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707958548369 time 708053963935 state 0x2 ref 0 00000100:00100000:3.0:1789760726.814618:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.814738:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200384 00000100:00100000:2.0:1789760726.814749:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200384 00000100:00100000:2.0:1789760726.814806:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 707958548369 time 708053963935 state 0x0 ref 1 00000100:00100000:2.0:1789760726.814823:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708054224351 time 708053963935 state 0x1 ref 1 00000100:00100000:2.0:1789760726.814856:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708054224351 time 708130158617 state 0x0 ref 1 00000100:00100000:2.0:1789760726.814872:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507200384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.814961:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18288640, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:2.0:1789760726.814970:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.814986:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760726.814990:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.814997:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.815011:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.859305:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971785 is committed 00002000:00010000:3.0:1789760726.859548:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18288640 -> 18292736 00010000:00010000:3.0:1789760726.859568:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859581:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859590:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859597:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.859602:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859612:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859636:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.859645:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7652 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.859823:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507200384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44946us (45208us total) trans 4294971785 rc 0/0 00000100:00100000:3.0:1789760726.859835:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4495 00000100:00100000:3.0:1789760726.859845:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708054224351 time 708130158617 state 0x2 ref 0 00000100:00100000:2.0:1789760726.863406:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760726.863513:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200512 00000100:00100000:1.0:1789760726.863523:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200512 00000100:00100000:1.0:1789760726.863582:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708054224351 time 708130158617 state 0x0 ref 1 00000100:00100000:1.0:1789760726.863596:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708130419033 time 708130158617 state 0x1 ref 1 00000100:00100000:1.0:1789760726.863661:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708130419033 time 708178963820 state 0x0 ref 1 00000100:00100000:1.0:1789760726.863673:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507200512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760726.863762:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18292736, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:1.0:1789760726.863771:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.863793:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760726.863798:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.863804:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.863819:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.906455:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18292736 -> 18296832 00010000:00010000:1.0:1789760726.906475:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906494:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906503:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906510:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760726.906515:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906524:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906543:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.906550:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7659 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18292736->18296831] (req 18292736->18296831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.906639:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507200512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42962us (43298us total) trans 4294971786 rc 0/0 00000100:00100000:1.0:1789760726.906649:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4496 00000100:00100000:1.0:1789760726.906658:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708130419033 time 708178963820 state 0x2 ref 0 00000001:00080000:1.0:1789760726.906732:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971786 is committed 00000100:00100000:1.0:1789760726.915675:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760726.915869:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200640 00000100:00100000:3.0:1789760726.915880:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200640 00000100:00100000:3.0:1789760726.915960:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708130419033 time 708178963820 state 0x0 ref 1 00000100:00100000:3.0:1789760726.915977:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708179224236 time 708178963820 state 0x1 ref 1 00000100:00100000:3.0:1789760726.916007:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708179224236 time 708231310374 state 0x0 ref 1 00000100:00100000:3.0:1789760726.916019:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507200640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760726.916066:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18296832, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760726.916075:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.916090:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760726.916095:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.916102:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.916116:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.941319:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760726.944828:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971787 is committed 00002000:00010000:3.0:1789760726.945019:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18296832 -> 18300928 00010000:00010000:3.0:1789760726.945033:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945045:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945053:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945061:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760726.945066:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945076:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945095:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.945108:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7660 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.945190:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507200640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29169us (29522us total) trans 4294971787 rc 0/0 00000100:00100000:3.0:1789760726.945200:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4497 00000100:00100000:3.0:1789760726.945618:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708179224236 time 708231310374 state 0x2 ref 0 00000100:00100000:3.0:1789760726.949578:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760726.949829:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200768 00000100:00100000:2.0:1789760726.949841:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200768 00000100:00100000:2.0:1789760726.949910:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708179224236 time 708231310374 state 0x0 ref 1 00000100:00100000:2.0:1789760726.949927:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708231570790 time 708231310374 state 0x1 ref 1 00000100:00100000:2.0:1789760726.949963:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708231570790 time 708265266213 state 0x0 ref 1 00000100:00100000:2.0:1789760726.949977:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507200768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760726.950024:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18300928, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:2.0:1789760726.950033:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.950050:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760726.950054:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.950060:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.950075:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.978624:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971788 is committed 00002000:00010000:2.0:1789760726.978852:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18300928 -> 18305024 00010000:00010000:2.0:1789760726.978867:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.978880:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.978889:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.978905:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760726.978911:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.978921:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.979006:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.979016:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7667 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.979151:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507200768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29172us (29577us total) trans 4294971788 rc 0/0 00000100:00100000:2.0:1789760726.979162:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4498 00000100:00100000:2.0:1789760726.979173:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708231570790 time 708265266213 state 0x2 ref 0 00000100:00100000:2.0:1789760726.982640:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760726.982735:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507200896 00000100:00100000:1.0:1789760726.982746:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507200896 00000100:00100000:1.0:1789760726.982806:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708231570790 time 708265266213 state 0x0 ref 1 00000100:00100000:1.0:1789760726.982822:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708265526629 time 708265266213 state 0x1 ref 1 00000100:00100000:1.0:1789760726.982856:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708265526629 time 708298159530 state 0x0 ref 1 00000100:00100000:1.0:1789760726.982868:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507200896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760726.982937:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18305024, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:1.0:1789760726.982946:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.982962:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760726.982967:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.982974:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.982988:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.016812:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18305024 -> 18309120 00010000:00010000:3.0:1789760727.016831:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016845:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016855:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016862:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.016867:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016876:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016910:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.016919:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b766e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18305024->18309119] (req 18305024->18309119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.017017:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507200896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34147us (34383us total) trans 4294971789 rc 0/0 00000100:00100000:3.0:1789760727.017027:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4499 00000100:00100000:3.0:1789760727.017036:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708265526629 time 708298159530 state 0x2 ref 0 00000001:00080000:0.0:1789760727.018860:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971789 is committed 00000100:00100000:1.0:1789760727.021265:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.021355:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201024 00000100:00100000:3.0:1789760727.021364:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201024 00000100:00100000:3.0:1789760727.021433:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708265526629 time 708298159530 state 0x0 ref 1 00000100:00100000:3.0:1789760727.021449:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708298419946 time 708298159530 state 0x1 ref 1 00000100:00100000:3.0:1789760727.021483:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708298419946 time 708336785993 state 0x0 ref 1 00000100:00100000:3.0:1789760727.021495:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.021540:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18309120, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9007 00010000:00010000:3.0:1789760727.021549:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.021564:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.021570:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.021577:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.021590:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760727.021797:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760727.047752:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18309120 -> 18313216 00002000:00010000:2.0:1789760727.047761:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:2.0:1789760727.047764:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:2.0:1789760727.047777:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047789:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047799:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047807:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760727.047812:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047828:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047847:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.047855:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7675 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.047971:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507201024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26473us (26710us total) trans 4294971790 rc 0/0 00000100:00100000:2.0:1789760727.047981:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4500 00000100:00100000:2.0:1789760727.047991:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708298419946 time 708336785993 state 0x2 ref 0 00000001:00080000:0.0:1789760727.049051:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971790 is committed 00000100:00100000:2.0:1789760727.051760:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.051848:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201152 00000100:00100000:3.0:1789760727.051858:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201152 00000100:00100000:3.0:1789760727.051923:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708298419946 time 708336785993 state 0x0 ref 1 00000100:00100000:3.0:1789760727.051949:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708337046409 time 708336785993 state 0x1 ref 1 00000100:00100000:0.0:1789760727.052014:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708337046409 time 708367315093 state 0x0 ref 1 00000100:00100000:0.0:1789760727.052030:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507201152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760727.052083:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18313216, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:0.0:1789760727.052092:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.052109:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760727.052114:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.052121:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.052136:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.079081:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971791 is committed 00002000:00010000:0.0:1789760727.079147:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18313216 -> 18317312 00010000:00010000:0.0:1789760727.079162:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079175:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079185:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079193:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760727.079198:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079207:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079227:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079235:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b767c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.079322:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507201152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27292us (27566us total) trans 4294971791 rc 0/0 00000100:00100000:0.0:1789760727.079333:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4501 00000100:00100000:0.0:1789760727.079572:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708337046409 time 708367315093 state 0x2 ref 0 00000100:00100000:2.0:1789760727.083834:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.084005:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201280 00000100:00100000:3.0:1789760727.084016:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201280 00000100:00100000:3.0:1789760727.084070:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708337046409 time 708367315093 state 0x0 ref 1 00000100:00100000:3.0:1789760727.084086:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708367575509 time 708367315093 state 0x1 ref 1 00000100:00100000:3.0:1789760727.084148:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708367575509 time 708399450790 state 0x0 ref 1 00000100:00100000:3.0:1789760727.084161:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.084208:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18317312, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.084216:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.084231:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.084236:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.084243:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.084257:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.113624:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971792 is committed 00002000:00010000:3.0:1789760727.113804:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18317312 -> 18321408 00010000:00010000:3.0:1789760727.113819:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113831:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113841:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113849:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.113854:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113864:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113885:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.113893:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7683 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.114103:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29940us (30275us total) trans 4294971792 rc 0/0 00000100:00100000:3.0:1789760727.114114:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4502 00000100:00100000:3.0:1789760727.114123:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708367575509 time 708399450790 state 0x2 ref 0 00000100:00100000:0.0:1789760727.117253:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.117326:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201408 00000100:00100000:3.0:1789760727.117336:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201408 00000100:00100000:3.0:1789760727.117402:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708367575509 time 708399450790 state 0x0 ref 1 00000100:00100000:3.0:1789760727.117419:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708399711206 time 708399450790 state 0x1 ref 1 00000100:00100000:3.0:1789760727.117450:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708399711206 time 708432752906 state 0x0 ref 1 00000100:00100000:3.0:1789760727.117462:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.117510:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18321408, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.117519:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.117536:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.117541:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.117548:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.117563:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.145636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971793 is committed 00002000:00010000:3.0:1789760727.145768:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18321408 -> 18325504 00010000:00010000:3.0:1789760727.145785:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145798:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145807:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145814:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.145819:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145829:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145848:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.145855:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b768a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.146074:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28607us (28826us total) trans 4294971793 rc 0/0 00000100:00100000:3.0:1789760727.146088:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4503 00000100:00100000:3.0:1789760727.146098:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708399711206 time 708432752906 state 0x2 ref 0 00000100:00100000:1.0:1789760727.150209:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.150453:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201536 00000100:00100000:3.0:1789760727.150463:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201536 00000100:00100000:3.0:1789760727.150520:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708399711206 time 708432752906 state 0x0 ref 1 00000100:00100000:3.0:1789760727.150536:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708433013322 time 708432752906 state 0x1 ref 1 00000100:00100000:3.0:1789760727.150565:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708433013322 time 708465868497 state 0x0 ref 1 00000100:00100000:3.0:1789760727.150577:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507201536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.150623:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18325504, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.150631:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.150646:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.150651:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.150657:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.150671:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.175154:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18325504 -> 18329600 00010000:00010000:3.0:1789760727.175185:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175199:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175210:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175218:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.175223:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175232:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175254:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.175262:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7691 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.175361:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507201536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24781us (25156us total) trans 4294971794 rc 0/0 00000100:00100000:3.0:1789760727.175371:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4504 00000100:00100000:3.0:1789760727.175380:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708433013322 time 708465868497 state 0x2 ref 0 00000001:00080000:1.0:1789760727.178215:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971794 is committed 00000100:00100000:0.0:1789760727.179266:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.179333:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201664 00000100:00100000:3.0:1789760727.179351:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201664 00000100:00100000:3.0:1789760727.179408:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708433013322 time 708465868497 state 0x0 ref 1 00000100:00100000:3.0:1789760727.179422:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708466128913 time 708465868497 state 0x1 ref 1 00000100:00100000:3.0:1789760727.179451:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708466128913 time 708494755169 state 0x0 ref 1 00000100:00100000:3.0:1789760727.179464:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507201664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.179511:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18329600, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.179519:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.179536:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.179540:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.179546:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.179559:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.208344:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971795 is committed 00002000:00010000:0.0:1789760727.208579:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18329600 -> 18333696 00010000:00010000:0.0:1789760727.208596:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208609:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208620:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208628:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760727.208633:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208643:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208668:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.208675:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7698 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.208806:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507201664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29340us (29541us total) trans 4294971795 rc 0/0 00000100:00100000:0.0:1789760727.208817:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4505 00000100:00100000:0.0:1789760727.208827:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708466128913 time 708494755169 state 0x2 ref 0 00000100:00100000:0.0:1789760727.212826:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.213008:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201792 00000100:00100000:3.0:1789760727.213017:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201792 00000100:00100000:3.0:1789760727.213071:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708466128913 time 708494755169 state 0x0 ref 1 00000100:00100000:3.0:1789760727.213096:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708495015585 time 708494755169 state 0x1 ref 1 00000100:00100000:3.0:1789760727.213133:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708495015585 time 708528436466 state 0x0 ref 1 00000100:00100000:3.0:1789760727.213153:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507201792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.213200:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18333696, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.213209:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.213224:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.213228:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.213235:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.213249:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.240159:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971796 is committed 00002000:00010000:2.0:1789760727.240256:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18333696 -> 18337792 00010000:00010000:2.0:1789760727.240271:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240283:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240292:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240305:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760727.240310:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240319:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240339:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240348:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b769f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.240443:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507201792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27288us (27620us total) trans 4294971796 rc 0/0 00000100:00100000:2.0:1789760727.240454:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4506 00000100:00100000:2.0:1789760727.240464:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708495015585 time 708528436466 state 0x2 ref 0 00000100:00100000:2.0:1789760727.243299:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.243380:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507201920 00000100:00100000:3.0:1789760727.243391:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507201920 00000100:00100000:3.0:1789760727.243445:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708495015585 time 708528436466 state 0x0 ref 1 00000100:00100000:3.0:1789760727.243461:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708528696882 time 708528436466 state 0x1 ref 1 00000100:00100000:3.0:1789760727.243540:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708528696882 time 708558842498 state 0x0 ref 1 00000100:00100000:3.0:1789760727.243554:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507201920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.243600:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18337792, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.243608:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.243626:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.243631:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.243637:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.243650:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.269915:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18337792 -> 18341888 00010000:00010000:1.0:1789760727.269945:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.269959:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.269970:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.269977:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.269982:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.269992:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.270012:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.270020:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.270120:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507201920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26562us (26824us total) trans 4294971797 rc 0/0 00000100:00100000:1.0:1789760727.270131:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4507 00000100:00100000:1.0:1789760727.270141:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708528696882 time 708558842498 state 0x2 ref 0 00000001:00080000:2.0:1789760727.270626:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971797 is committed 00000100:00100000:0.0:1789760727.279482:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.279573:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202048 00000100:00100000:3.0:1789760727.279583:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202048 00000100:00100000:3.0:1789760727.279636:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708528696882 time 708558842498 state 0x0 ref 1 00000100:00100000:3.0:1789760727.279653:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708559102914 time 708558842498 state 0x1 ref 1 00000100:00100000:3.0:1789760727.279681:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708559102914 time 708594984761 state 0x0 ref 1 00000100:00100000:3.0:1789760727.279734:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507202048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.279783:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18341888, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.279792:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.279808:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.279813:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.279819:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.279839:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.315647:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971798 is committed 00002000:00010000:0.0:1789760727.315745:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18341888 -> 18345984 00010000:00010000:0.0:1789760727.315760:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315774:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315784:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315792:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760727.315797:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315807:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315828:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.315837:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.316043:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507202048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36345us (36566us total) trans 4294971798 rc 0/0 00000100:00100000:0.0:1789760727.316054:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4508 00000100:00100000:0.0:1789760727.316065:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708559102914 time 708594984761 state 0x2 ref 0 00000100:00100000:0.0:1789760727.324109:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760727.324235:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202176 00000100:00100000:2.0:1789760727.324245:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202176 00000100:00100000:2.0:1789760727.324302:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708559102914 time 708594984761 state 0x0 ref 1 00000100:00100000:2.0:1789760727.324318:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708595245177 time 708594984761 state 0x1 ref 1 00000100:00100000:2.0:1789760727.324386:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708595245177 time 708639689019 state 0x0 ref 1 00000100:00100000:2.0:1789760727.324400:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507202176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760727.324452:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18345984, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:2.0:1789760727.324462:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.324475:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760727.324479:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.324486:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.324500:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.348470:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18345984 -> 18350080 00010000:00010000:1.0:1789760727.348490:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348504:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348514:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348523:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.348528:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348539:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348559:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.348568:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b76b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18345984->18350079] (req 18345984->18350079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.348665:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507202176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24262us (24560us total) trans 4294971799 rc 0/0 00000100:00100000:1.0:1789760727.348676:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4509 00000100:00100000:1.0:1789760727.348686:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708595245177 time 708639689019 state 0x2 ref 0 00000001:00080000:1.0:1789760727.348752:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971799 is committed 00000100:00100000:2.0:1789760727.352995:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.353107:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202304 00000100:00100000:1.0:1789760727.353116:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202304 00000100:00100000:1.0:1789760727.353169:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708595245177 time 708639689019 state 0x0 ref 1 00000100:00100000:1.0:1789760727.353184:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708639949435 time 708639689019 state 0x1 ref 1 00000100:00100000:1.0:1789760727.353215:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708639949435 time 708668518684 state 0x0 ref 1 00000100:00100000:1.0:1789760727.353228:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507202304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.353278:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18350080, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.353288:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.353304:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760727.353309:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.353315:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.353330:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.382554:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971800 is committed 00002000:00010000:1.0:1789760727.382650:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18350080 -> 18354176 00010000:00010000:1.0:1789760727.382665:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382677:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382686:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382724:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.382730:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382739:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382760:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.382768:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b76bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.382849:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507202304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29618us (29858us total) trans 4294971800 rc 0/0 00000100:00100000:1.0:1789760727.382859:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4510 00000100:00100000:1.0:1789760727.382868:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708639949435 time 708668518684 state 0x2 ref 0 00000100:00100000:0.0:1789760727.386559:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.386656:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202432 00000100:00100000:3.0:1789760727.386664:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202432 00000100:00100000:3.0:1789760727.386782:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708639949435 time 708668518684 state 0x0 ref 1 00000100:00100000:3.0:1789760727.386797:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708668779100 time 708668518684 state 0x1 ref 1 00000100:00100000:3.0:1789760727.386862:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708668779100 time 708702164138 state 0x0 ref 1 00000100:00100000:3.0:1789760727.386876:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507202432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.386998:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18354176, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.387008:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.387024:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.387029:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.387035:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.387049:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.417375:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18354176 -> 18358272 00010000:00010000:0.0:1789760727.417399:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417412:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417422:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417430:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760727.417436:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417447:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417467:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.417476:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.417579:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507202432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30700us (31024us total) trans 4294971801 rc 0/0 00000100:00100000:0.0:1789760727.417589:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4511 00000100:00100000:0.0:1789760727.417600:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708668779100 time 708702164138 state 0x2 ref 0 00000001:00080000:2.0:1789760727.420808:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971801 is committed 00000100:00100000:2.0:1789760727.421671:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.421889:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202560 00000100:00100000:1.0:1789760727.421909:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202560 00000100:00100000:1.0:1789760727.422029:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708668779100 time 708702164138 state 0x0 ref 1 00000100:00100000:1.0:1789760727.422046:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708702424554 time 708702164138 state 0x1 ref 1 00000100:00100000:1.0:1789760727.422117:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708702424554 time 708737418891 state 0x0 ref 1 00000100:00100000:1.0:1789760727.422132:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.422181:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18358272, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.422191:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.422207:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760727.422211:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.422218:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.422232:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.447847:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971802 is committed 00002000:00010000:2.0:1789760727.448024:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18358272 -> 18362368 00010000:00010000:2.0:1789760727.448041:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448054:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448068:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448075:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760727.448080:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448089:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448108:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.448117:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b76c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.448213:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26079us (26543us total) trans 4294971802 rc 0/0 00000100:00100000:2.0:1789760727.448223:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4512 00000100:00100000:2.0:1789760727.448232:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708702424554 time 708737418891 state 0x2 ref 0 00000100:00100000:1.0:1789760727.451842:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.451932:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202688 00000100:00100000:3.0:1789760727.451950:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202688 00000100:00100000:3.0:1789760727.452000:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708702424554 time 708737418891 state 0x0 ref 1 00000100:00100000:3.0:1789760727.452017:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708737679307 time 708737418891 state 0x1 ref 1 00000100:00100000:3.0:1789760727.452046:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708737679307 time 708767349435 state 0x0 ref 1 00000100:00100000:3.0:1789760727.452059:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.452106:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18362368, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.452115:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.452130:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.452136:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.452142:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.452157:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.496626:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971803 is committed 00002000:00010000:3.0:1789760727.496774:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18362368 -> 18366464 00010000:00010000:3.0:1789760727.496790:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496802:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496817:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496825:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.496831:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496840:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496861:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.496870:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.497089:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45026us (45250us total) trans 4294971803 rc 0/0 00000100:00100000:3.0:1789760727.497101:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4513 00000100:00100000:3.0:1789760727.497110:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708737679307 time 708767349435 state 0x2 ref 0 00000100:00100000:0.0:1789760727.500502:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.500593:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202816 00000100:00100000:3.0:1789760727.500603:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202816 00000100:00100000:3.0:1789760727.500656:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708737679307 time 708767349435 state 0x0 ref 1 00000100:00100000:3.0:1789760727.500672:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708767609851 time 708767349435 state 0x1 ref 1 00000100:00100000:3.0:1789760727.500805:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708767609851 time 708816106191 state 0x0 ref 1 00000100:00100000:3.0:1789760727.500819:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.500868:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18366464, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.500876:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.500902:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.500908:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.500915:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.500997:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.554113:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971804 is committed 00002000:00010000:1.0:1789760727.554198:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18366464 -> 18370560 00010000:00010000:1.0:1789760727.554213:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554231:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554240:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554248:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.554253:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554261:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554281:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554289:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b76d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.554497:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507202816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53675us (54000us total) trans 4294971804 rc 0/0 00000100:00100000:1.0:1789760727.554509:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4514 00000100:00100000:1.0:1789760727.554518:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708767609851 time 708816106191 state 0x2 ref 0 00000100:00100000:2.0:1789760727.558234:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.558325:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507202944 00000100:00100000:1.0:1789760727.558335:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507202944 00000100:00100000:1.0:1789760727.558453:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708767609851 time 708816106191 state 0x0 ref 1 00000100:00100000:1.0:1789760727.558468:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708816366607 time 708816106191 state 0x1 ref 1 00000100:00100000:1.0:1789760727.558541:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708816366607 time 708873842955 state 0x0 ref 1 00000100:00100000:1.0:1789760727.558554:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507202944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.558605:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18370560, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.558614:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.558630:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760727.558635:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.558641:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.558655:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.610846:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971805 is committed 00002000:00010000:1.0:1789760727.610988:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18370560 -> 18374656 00010000:00010000:1.0:1789760727.611004:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611018:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611032:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611040:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.611045:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611055:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611076:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.611083:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.611216:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507202944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52658us (52986us total) trans 4294971805 rc 0/0 00000100:00100000:1.0:1789760727.611227:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4515 00000100:00100000:1.0:1789760727.611236:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708816366607 time 708873842955 state 0x2 ref 0 00000100:00100000:2.0:1789760727.616096:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.616198:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203072 00000100:00100000:3.0:1789760727.616208:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203072 00000100:00100000:3.0:1789760727.616264:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708816366607 time 708873842955 state 0x0 ref 1 00000100:00100000:3.0:1789760727.616279:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708874103371 time 708873842955 state 0x1 ref 1 00000100:00100000:3.0:1789760727.616353:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708874103371 time 708931656466 state 0x0 ref 1 00000100:00100000:3.0:1789760727.616366:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.616414:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18374656, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.616424:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.616440:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760727.616445:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.616452:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.616478:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.654758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971806 is committed 00002000:00010000:3.0:1789760727.655036:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18374656 -> 18378752 00010000:00010000:3.0:1789760727.655052:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655065:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655074:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655081:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.655086:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655095:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655119:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.655127:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.655285:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38917us (39194us total) trans 4294971806 rc 0/0 00000100:00100000:3.0:1789760727.655297:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4516 00000100:00100000:3.0:1789760727.655307:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708874103371 time 708931656466 state 0x2 ref 0 00000100:00100000:2.0:1789760727.659571:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.659640:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203200 00000100:00100000:1.0:1789760727.659651:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203200 00000100:00100000:1.0:1789760727.659734:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708874103371 time 708931656466 state 0x0 ref 1 00000100:00100000:1.0:1789760727.659755:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708931916882 time 708931656466 state 0x1 ref 1 00000100:00100000:1.0:1789760727.659835:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708931916882 time 708975137579 state 0x0 ref 1 00000100:00100000:1.0:1789760727.659848:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.659902:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18378752, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.659911:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.659925:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760727.659936:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.659943:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.659958:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.693639:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971807 is committed 00002000:00010000:1.0:1789760727.693728:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18378752 -> 18382848 00010000:00010000:1.0:1789760727.693743:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693756:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693766:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693775:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.693780:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693789:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693880:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.693889:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b76ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.693995:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34144us (34428us total) trans 4294971807 rc 0/0 00000100:00100000:1.0:1789760727.694007:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4517 00000100:00100000:1.0:1789760727.694017:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708931916882 time 708975137579 state 0x2 ref 0 00000100:00100000:1.0:1789760727.698541:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.698627:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203328 00000100:00100000:3.0:1789760727.698637:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203328 00000100:00100000:3.0:1789760727.698731:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708931916882 time 708975137579 state 0x0 ref 1 00000100:00100000:3.0:1789760727.698748:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708975397995 time 708975137579 state 0x1 ref 1 00000100:00100000:3.0:1789760727.698769:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708975397995 time 709014072554 state 0x0 ref 1 00000100:00100000:3.0:1789760727.698783:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.698964:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18382848, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.698974:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.698989:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760727.698994:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.699000:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.699014:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.736066:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971808 is committed 00002000:00010000:3.0:1789760727.736187:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18382848 -> 18386944 00010000:00010000:3.0:1789760727.736203:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736216:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736225:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736233:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.736237:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736246:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736267:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.736279:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b76f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.736485:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507203328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37699us (37949us total) trans 4294971808 rc 0/0 00000100:00100000:3.0:1789760727.736496:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4518 00000100:00100000:3.0:1789760727.736506:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708975397995 time 709014072554 state 0x2 ref 0 00000100:00100000:2.0:1789760727.740531:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.740622:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203456 00000100:00100000:3.0:1789760727.740633:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203456 00000100:00100000:3.0:1789760727.740688:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 708975397995 time 709014072554 state 0x0 ref 1 00000100:00100000:3.0:1789760727.741068:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709014332970 time 709014072554 state 0x1 ref 1 00000100:00100000:3.0:1789760727.741106:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709014332970 time 709056409035 state 0x0 ref 1 00000100:00100000:3.0:1789760727.741118:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507203456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.741168:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18386944, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.741177:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741192:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760727.741197:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.741203:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741221:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.768752:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971809 is committed 00000100:00100000:1.0:1789760727.769445:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.769525:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.769607:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.769637:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760727.769678:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203584 00000100:00100000:2.0:1789760727.769687:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203584 00000100:00100000:2.0:1789760727.769756:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 462 00000100:00100000:2.0:1789760727.769767:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223496c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507203584:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760727.769856:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223496c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507203584:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 91us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.769863:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 462 00000100:00100000:0.0:1789760727.770965:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203712 00000100:00100000:0.0:1789760727.770974:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203712 00000100:00100000:0.0:1789760727.771025:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203968 00000100:00100000:0.0:1789760727.771029:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203968 00000100:00100000:0.0:1789760727.771058:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 858 00000100:00100000:0.0:1789760727.771066:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507203712:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760727.771138:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507203712:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 72us (1700us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760727.771145:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 858 00000100:00100000:0.0:1789760727.771160:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 859 00000100:00100000:0.0:1789760727.771167:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507203968:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760727.771192:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507203968:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 26us (1667us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760727.771199:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 859 00000100:00080000:2.0:1789760727.771570:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760727.772751:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00002000:00010000:1.0:1789760727.772804:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18386944 -> 18391040 00010000:00010000:1.0:1789760727.772829:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772841:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772850:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772857:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760727.772861:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772871:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772898:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.772905:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b76fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.772974:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507203456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31857us (32452us total) trans 4294971809 rc 0/0 00000100:00100000:1.0:1789760727.772984:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4519 00000100:00100000:1.0:1789760727.772993:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709014332970 time 709056409035 state 0x2 ref 0 00000100:00100000:2.0:1789760727.773172:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507203840 00000100:00100000:2.0:1789760727.773180:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507203840 00000100:00100000:2.0:1789760727.773230:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204096 00000100:00100000:2.0:1789760727.773233:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204096 00000100:00100000:2.0:1789760727.773254:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 860 00000100:00100000:2.0:1789760727.773262:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507203840:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760727.773302:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507203840:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 41us (3664us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.773309:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 860 00000100:00100000:2.0:1789760727.773322:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 861 00000100:00100000:2.0:1789760727.773328:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507204096:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760727.773365:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507204096:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 36us (619us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.773371:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 861 00000100:00080000:1.0:1789760727.774537:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760727.777510:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.778045:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204224 00000100:00100000:1.0:1789760727.778056:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204224 00000100:00100000:1.0:1789760727.778111:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709014332970 time 709056409035 state 0x0 ref 1 00000100:00100000:1.0:1789760727.778127:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709056669451 time 709056409035 state 0x1 ref 1 00000100:00100000:1.0:1789760727.778193:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709056669451 time 709093495666 state 0x0 ref 1 00000100:00100000:1.0:1789760727.778205:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507204224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.778251:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18391040, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.778258:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.778272:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760727.778277:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.778283:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.778299:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.817233:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18391040 -> 18395136 00010000:00010000:2.0:1789760727.817252:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817271:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817281:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817288:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760727.817293:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817301:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817322:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.817330:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7701 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.817438:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507204224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39229us (39931us total) trans 4294971810 rc 0/0 00000100:00100000:2.0:1789760727.817447:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4520 00000100:00100000:2.0:1789760727.817456:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709056669451 time 709093495666 state 0x2 ref 0 00000001:00080000:3.0:1789760727.821412:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971810 is committed 00000100:00100000:2.0:1789760727.821665:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760727.821912:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204352 00000100:00100000:3.0:1789760727.821922:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204352 00000100:00100000:3.0:1789760727.822041:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709056669451 time 709093495666 state 0x0 ref 1 00000100:00100000:3.0:1789760727.822056:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709093756082 time 709093495666 state 0x1 ref 1 00000100:00100000:3.0:1789760727.822116:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709093756082 time 709137419114 state 0x0 ref 1 00000100:00100000:3.0:1789760727.822130:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507204352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760727.822183:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18395136, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:3.0:1789760727.822193:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.822208:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760727.822213:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.822219:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.822233:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.855762:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971811 is committed 00002000:00010000:3.0:1789760727.856007:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18395136 -> 18399232 00010000:00010000:3.0:1789760727.856023:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856036:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856047:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856054:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760727.856059:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856069:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856089:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.856097:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7708 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.856233:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507204352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34100us (34567us total) trans 4294971811 rc 0/0 00000100:00100000:3.0:1789760727.856248:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4521 00000100:00100000:3.0:1789760727.856258:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709093756082 time 709137419114 state 0x2 ref 0 00000100:00100000:0.0:1789760727.860620:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760727.860914:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204480 00000100:00100000:1.0:1789760727.860924:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204480 00000100:00100000:1.0:1789760727.861034:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709093756082 time 709137419114 state 0x0 ref 1 00000100:00100000:1.0:1789760727.861049:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709137679530 time 709137419114 state 0x1 ref 1 00000100:00100000:1.0:1789760727.861114:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709137679530 time 709176416996 state 0x0 ref 1 00000100:00100000:1.0:1789760727.861127:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507204480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760727.861178:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18399232, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:1.0:1789760727.861187:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.861202:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760727.861206:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.861213:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.861228:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.934444:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18399232 -> 18403328 00010000:00010000:0.0:1789760727.934465:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934479:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934488:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934495:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760727.934499:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934514:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934534:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.934541:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b770f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.934684:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507204480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73553us (74067us total) trans 4294971812 rc 0/0 00000001:00080000:2.0:1789760727.934725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971812 is committed 00000100:00100000:0.0:1789760727.934772:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4522 00000100:00100000:0.0:1789760727.934782:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709137679530 time 709176416996 state 0x2 ref 0 00000100:00100000:3.0:1789760727.938826:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760727.938952:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204608 00000100:00100000:2.0:1789760727.938962:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204608 00000100:00100000:2.0:1789760727.939016:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709137679530 time 709176416996 state 0x0 ref 1 00000100:00100000:2.0:1789760727.939030:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709176677412 time 709176416996 state 0x1 ref 1 00000100:00100000:0.0:1789760727.939094:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709176677412 time 709254394016 state 0x0 ref 1 00000100:00100000:0.0:1789760727.939115:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507204608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760727.939166:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18403328, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:0.0:1789760727.939175:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.939190:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760727.939195:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.939201:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.939215:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.965602:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760728.012564:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971813 is committed 00002000:00010000:0.0:1789760728.012650:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18403328 -> 18407424 00010000:00010000:0.0:1789760728.012667:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012680:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012688:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012715:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.012720:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012728:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012746:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.012754:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7716 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.012865:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507204608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73753us (74055us total) trans 4294971813 rc 0/0 00000100:00100000:0.0:1789760728.012876:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4523 00000100:00100000:0.0:1789760728.012885:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709176677412 time 709254394016 state 0x2 ref 0 00000100:00100000:3.0:1789760728.016830:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.019954:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204736 00000100:00100000:0.0:1789760728.019970:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204736 00000100:00100000:0.0:1789760728.020035:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709176677412 time 709254394016 state 0x0 ref 1 00000100:00100000:0.0:1789760728.020051:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709254654432 time 709254394016 state 0x1 ref 1 00000100:00100000:0.0:1789760728.020087:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709254654432 time 709335389677 state 0x0 ref 1 00000100:00100000:0.0:1789760728.020101:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507204736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.020160:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18407424, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9007 00010000:00010000:0.0:1789760728.020170:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.020201:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.020205:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.020212:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.020226:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.070620:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971814 is committed 00002000:00010000:0.0:1789760728.070740:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18407424 -> 18411520 00002000:00010000:0.0:1789760728.070745:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:0.0:1789760728.070748:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:0.0:1789760728.070761:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070774:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070784:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070791:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.070796:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070804:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070839:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.070846:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b771d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.071041:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507204736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50939us (54215us total) trans 4294971814 rc 0/0 00000100:00100000:0.0:1789760728.071052:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4524 00000100:00100000:0.0:1789760728.071061:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709254654432 time 709335389677 state 0x2 ref 0 00000100:00100000:2.0:1789760728.075620:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.076084:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204864 00000100:00100000:0.0:1789760728.076095:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204864 00000100:00100000:0.0:1789760728.076151:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709254654432 time 709335389677 state 0x0 ref 1 00000100:00100000:0.0:1789760728.076166:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709335650093 time 709335389677 state 0x1 ref 1 00000100:00100000:0.0:1789760728.076237:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709335650093 time 709391539460 state 0x0 ref 1 00000100:00100000:0.0:1789760728.076250:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507204864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.076299:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18411520, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.076308:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076324:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.076329:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.076345:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076360:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.112495:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971815 is committed 00002000:00010000:1.0:1789760728.112595:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18411520 -> 18415616 00010000:00010000:1.0:1789760728.112611:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112624:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112634:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112646:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.112651:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112661:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112680:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.112688:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7724 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.112806:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507204864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36554us (37191us total) trans 4294971815 rc 0/0 00000100:00100000:1.0:1789760728.112816:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4525 00000100:00100000:1.0:1789760728.112826:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709335650093 time 709391539460 state 0x2 ref 0 00000100:00100000:1.0:1789760728.117302:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.117431:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507204992 00000100:00100000:0.0:1789760728.117443:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507204992 00000100:00100000:0.0:1789760728.117497:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709335650093 time 709391539460 state 0x0 ref 1 00000100:00100000:0.0:1789760728.117512:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709391799876 time 709391539460 state 0x1 ref 1 00000100:00100000:0.0:1789760728.117584:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709391799876 time 709432886013 state 0x0 ref 1 00000100:00100000:0.0:1789760728.117597:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507204992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.117646:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18415616, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.117655:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.117671:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.117676:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.117682:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.117723:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.148602:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18415616 -> 18419712 00010000:00010000:0.0:1789760728.148624:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148643:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148652:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148659:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.148664:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148674:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148710:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.148719:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b772b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.148813:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507204992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31213us (31516us total) trans 4294971816 rc 0/0 00000100:00100000:0.0:1789760728.148823:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4526 00000100:00100000:0.0:1789760728.148832:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709391799876 time 709432886013 state 0x2 ref 0 00000001:00080000:0.0:1789760728.148870:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971816 is committed 00000100:00100000:2.0:1789760728.153273:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.153419:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205120 00000100:00100000:0.0:1789760728.153428:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205120 00000100:00100000:0.0:1789760728.153477:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709391799876 time 709432886013 state 0x0 ref 1 00000100:00100000:0.0:1789760728.153491:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709433146429 time 709432886013 state 0x1 ref 1 00000100:00100000:0.0:1789760728.153546:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709433146429 time 709468848757 state 0x0 ref 1 00000100:00100000:0.0:1789760728.153559:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.153602:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18419712, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.153611:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.153625:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.153629:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.153635:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.153648:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.182513:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18419712 -> 18423808 00010000:00010000:0.0:1789760728.182533:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182545:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182559:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182566:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.182571:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182580:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182599:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.182606:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7732 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.182759:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971817 is committed 00000100:00100000:2.0:1789760728.185828:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.185951:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29136us (29427us total) trans 4294971817 rc 0/0 00000100:00100000:0.0:1789760728.185965:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4527 00000100:00100000:0.0:1789760728.185974:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709433146429 time 709468848757 state 0x2 ref 0 00000100:00100000:0.0:1789760728.186006:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205248 00000100:00100000:0.0:1789760728.186015:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205248 00000100:00100000:0.0:1789760728.186059:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709433146429 time 709468848757 state 0x0 ref 1 00000100:00100000:0.0:1789760728.186072:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709469109173 time 709468848757 state 0x1 ref 1 00000100:00100000:0.0:1789760728.186092:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709469109173 time 709501395616 state 0x0 ref 1 00000100:00100000:0.0:1789760728.186102:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507205248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.186149:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18423808, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.186158:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.186173:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.186177:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.186184:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.186199:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.210548:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971818 is committed 00002000:00010000:1.0:1789760728.210731:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18423808 -> 18427904 00010000:00010000:1.0:1789760728.210747:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210761:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210776:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210783:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.210788:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210798:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210817:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.210825:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7739 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.210964:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507205248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24858us (25142us total) trans 4294971818 rc 0/0 00000100:00100000:1.0:1789760728.210975:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4528 00000100:00100000:1.0:1789760728.210985:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709469109173 time 709501395616 state 0x2 ref 0 00000100:00100000:2.0:1789760728.214869:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.215110:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205376 00000100:00100000:1.0:1789760728.215120:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205376 00000100:00100000:0.0:1789760728.215243:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709469109173 time 709501395616 state 0x0 ref 1 00000100:00100000:0.0:1789760728.215259:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709501656032 time 709501395616 state 0x1 ref 1 00000100:00100000:0.0:1789760728.215327:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709501656032 time 709530629624 state 0x0 ref 1 00000100:00100000:0.0:1789760728.215350:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.215397:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18427904, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.215406:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.215423:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.215428:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.215434:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.215447:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.244495:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18427904 -> 18432000 00010000:00010000:1.0:1789760728.244515:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244529:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244539:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244546:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.244551:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244560:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244580:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.244588:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7740 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.244683:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507205376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29331us (29817us total) trans 4294971819 rc 0/0 00000100:00100000:1.0:1789760728.244725:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4529 00000100:00100000:1.0:1789760728.244735:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709501656032 time 709530629624 state 0x2 ref 0 00000001:00080000:2.0:1789760728.244950:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971819 is committed 00000100:00100000:2.0:1789760728.248026:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760728.248179:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205504 00000100:00100000:3.0:1789760728.248200:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205504 00000100:00100000:3.0:1789760728.248260:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709501656032 time 709530629624 state 0x0 ref 1 00000100:00100000:3.0:1789760728.248277:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709530890040 time 709530629624 state 0x1 ref 1 00000100:00100000:3.0:1789760728.248313:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709530890040 time 709563616569 state 0x0 ref 1 00000100:00100000:3.0:1789760728.248327:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507205504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760728.248492:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18432000, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:3.0:1789760728.248501:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.248518:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760728.248523:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.248528:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.248548:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.281363:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18432000 -> 18436096 00010000:00010000:0.0:1789760728.281384:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281398:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281408:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281414:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.281419:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281427:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281445:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.281454:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7747 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.281553:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507205504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33223us (33531us total) trans 4294971820 rc 0/0 00000100:00100000:0.0:1789760728.281564:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4530 00000100:00100000:0.0:1789760728.281574:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709530890040 time 709563616569 state 0x2 ref 0 00000100:00100000:3.0:1789760728.284672:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760728.284712:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266752:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760728.284796:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.284866:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266752 00000100:00100000:3.0:1789760728.284874:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266752 00000100:00100000:3.0:1789760728.284948:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789760728.284959:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538266752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760728.284994:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760728.285042:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538266752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 85us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.285049:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:3.0:1789760728.285071:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760728.285109:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538266752:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760728.286061:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538267008:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.286109:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.286293:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538267520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.286318:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287266:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538268032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287303:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287345:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1789760728.287353:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538266880:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287385:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287406:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538267392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287431:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287452:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538267904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287476:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287510:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538267264:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287535:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287555:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538267776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287577:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287595:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538268288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287616:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287635:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1789760728.287641:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538267136:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287664:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287683:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538267648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287722:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760728.287740:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538268160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.287760:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760728.288168:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538266880 00000100:00100000:0.0:1789760728.288176:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538266880 00000100:00100000:0.0:1789760728.288226:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789760728.288236:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223495850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538266880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288293:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223495850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538266880:12345-0@lo:400:kworker.0 Request processed in 59us (906us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.288301:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789760728.288346:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267904 00000100:00100000:0.0:1789760728.288350:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267904 00000100:00100000:0.0:1789760728.288382:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267776 00000100:00100000:0.0:1789760728.288385:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267776 00000100:00100000:0.0:1789760728.288405:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268288 00000100:00100000:0.0:1789760728.288408:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268288 00000100:00100000:0.0:1789760728.288426:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1789760728.288433:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538267904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288466:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538267904:12345-0@lo:400:kworker.0 Request processed in 34us (989us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.288472:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1789760728.288486:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1789760728.288492:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538267776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288525:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538267776:12345-0@lo:400:kworker.0 Request processed in 34us (947us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.288532:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1789760728.288542:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789760728.288549:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538268288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288577:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538268288:12345-0@lo:400:kworker.0 Request processed in 28us (960us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.288582:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00080000:0.0:1789760728.288607:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760728.288611:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760728.288640:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538267776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288650:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538268288:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288676:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538266880:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.288685:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538267904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.289172:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267008 00000100:00100000:3.0:1789760728.289177:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267008 00000100:00100000:3.0:1789760728.289204:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267520 00000100:00100000:3.0:1789760728.289206:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267520 00000100:00100000:3.0:1789760728.289233:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268032 00000100:00100000:3.0:1789760728.289237:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268032 00000100:00100000:3.0:1789760728.289249:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267264 00000100:00100000:3.0:1789760728.289252:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267264 00000100:00100000:3.0:1789760728.289262:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267648 00000100:00100000:3.0:1789760728.289264:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267648 00000100:00100000:3.0:1789760728.289276:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268160 00000100:00100000:3.0:1789760728.289279:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268160 00000100:00100000:3.0:1789760728.289294:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 862 00000100:00100000:3.0:1789760728.289300:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538267008:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789760728.289470:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760728.289474:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760728.289509:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538267008:12345-0@lo:400:kworker.0 Request processed in 209us (3399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.289517:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 862 00000100:00100000:3.0:1789760728.289537:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 863 00000100:00100000:3.0:1789760728.289544:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538267520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.289575:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538267520:12345-0@lo:400:kworker.0 Request processed in 32us (3256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.289581:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 863 00000100:00100000:3.0:1789760728.289593:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 864 00000100:00100000:3.0:1789760728.289599:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538268032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.289627:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538268032:12345-0@lo:400:kworker.0 Request processed in 29us (2324us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.289634:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 864 00000100:00100000:3.0:1789760728.289646:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 865 00000100:00100000:3.0:1789760728.289652:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538267264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.289715:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538267264:12345-0@lo:400:kworker.0 Request processed in 40us (2155us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.289723:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 865 00000100:00100000:3.0:1789760728.289736:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1789760728.289742:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538267648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.289869:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538267648:12345-0@lo:400:kworker.0 Request processed in 126us (2145us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.289875:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1789760728.289899:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 867 00000100:00100000:3.0:1789760728.289905:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538268160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.290003:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538268160:12345-0@lo:400:kworker.0 Request processed in 97us (2241us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.290009:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 867 00000100:00100000:3.0:1789760728.290071:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760728.290102:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267392 00000100:00100000:3.0:1789760728.290106:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267392 00000100:00100000:3.0:1789760728.290206:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538267136 00000100:00100000:3.0:1789760728.290209:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538267136 00000100:00100000:3.0:1789760728.290225:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1789760728.290231:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538267392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.290265:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538267392:12345-0@lo:400:kworker.0 Request processed in 34us (2833us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.290272:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1789760728.290283:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:3.0:1789760728.290289:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538267136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.290317:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538267136:12345-0@lo:400:kworker.0 Request processed in 28us (2652us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760728.290323:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000001:00080000:2.0:1789760728.290446:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971820 is committed 00000100:00100000:0.0:1789760728.290670:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227828700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538268416:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760728.290724:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760728.290763:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268416 00000100:00100000:0.0:1789760728.290767:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268416 00000100:00100000:0.0:1789760728.290806:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1789760728.290813:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538268416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760728.290829:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760728.290865:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538268416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 52us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.290871:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1789760728.290920:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227828700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538268416:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760728.290964:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538267136:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.290976:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538267648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.290985:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538268160:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.290994:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538267264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.291560:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538267008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.291574:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538267520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.291585:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538268032:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.291606:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538267392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.291627:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205632 00000100:00100000:1.0:1789760728.291631:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205632 00000100:00100000:1.0:1789760728.291668:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709530890040 time 709563616569 state 0x0 ref 1 00000100:00100000:1.0:1789760728.291681:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709563876985 time 709563616569 state 0x1 ref 1 00000100:00100000:3.0:1789760728.291957:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709563876985 time 709607256957 state 0x0 ref 1 00000100:00100000:3.0:1789760728.291969:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760728.292010:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18436096, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:3.0:1789760728.292019:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.292036:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760728.292040:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.292047:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.292061:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.333985:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971821 is committed 00002000:00010000:0.0:1789760728.334055:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18436096 -> 18440192 00010000:00010000:0.0:1789760728.334070:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334083:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334092:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334098:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.334103:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334344:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334375:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334383:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b774e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.334488:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42517us (44414us total) trans 4294971821 rc 0/0 00000100:00100000:0.0:1789760728.334498:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4531 00000100:00100000:0.0:1789760728.334507:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709563876985 time 709607256957 state 0x2 ref 0 00000100:00100000:3.0:1789760728.341035:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.341802:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205760 00000100:00100000:1.0:1789760728.341813:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205760 00000100:00100000:1.0:1789760728.341875:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709563876985 time 709607256957 state 0x0 ref 1 00000100:00100000:1.0:1789760728.341901:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709607517373 time 709607256957 state 0x1 ref 1 00000100:00100000:1.0:1789760728.341922:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709607517373 time 709657225871 state 0x0 ref 1 00000100:00100000:1.0:1789760728.341934:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507205760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.341982:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18440192, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.341991:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342007:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760728.342013:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.342020:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342034:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.367493:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971822 is committed 00002000:00010000:1.0:1789760728.367595:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18440192 -> 18444288 00010000:00010000:1.0:1789760728.367611:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367623:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367631:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367638:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.367643:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367653:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367683:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.367709:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7755 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.367813:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507205760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25875us (26783us total) trans 4294971822 rc 0/0 00000100:00100000:1.0:1789760728.367825:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4532 00000100:00100000:1.0:1789760728.367834:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709607517373 time 709657225871 state 0x2 ref 0 00000100:00100000:2.0:1789760728.373909:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.376663:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507205888 00000100:00100000:0.0:1789760728.376678:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507205888 00000100:00100000:0.0:1789760728.376759:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709607517373 time 709657225871 state 0x0 ref 1 00000100:00100000:0.0:1789760728.376774:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709657486287 time 709657225871 state 0x1 ref 1 00000100:00100000:0.0:1789760728.376867:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709657486287 time 709692165030 state 0x0 ref 1 00000100:00100000:0.0:1789760728.376882:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507205888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.376949:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18444288, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.376958:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.376978:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.376983:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.376989:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.377002:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.407394:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18444288 -> 18448384 00010000:00010000:0.0:1789760728.407414:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407428:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407436:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407443:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.407448:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407457:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407478:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407486:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b775c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.407575:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507205888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30692us (33682us total) trans 4294971823 rc 0/0 00000100:00100000:0.0:1789760728.407585:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4533 00000100:00100000:0.0:1789760728.407595:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709657486287 time 709692165030 state 0x2 ref 0 00000001:00080000:0.0:1789760728.407631:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971823 is committed 00000100:00100000:3.0:1789760728.411546:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.411625:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206016 00000100:00100000:0.0:1789760728.411636:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206016 00000100:00100000:0.0:1789760728.411753:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709657486287 time 709692165030 state 0x0 ref 1 00000100:00100000:0.0:1789760728.411768:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709692425446 time 709692165030 state 0x1 ref 1 00000100:00100000:0.0:1789760728.411797:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709692425446 time 709727100523 state 0x0 ref 1 00000100:00100000:0.0:1789760728.411809:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507206016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.411857:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18448384, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.411865:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.411901:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.411906:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.411912:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.411927:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.437720:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971824 is committed 00002000:00010000:3.0:1789760728.437820:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18448384 -> 18452480 00010000:00010000:3.0:1789760728.437835:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437849:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437859:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437866:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760728.437871:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437881:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437908:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.437916:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7763 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.438077:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507206016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26264us (26534us total) trans 4294971824 rc 0/0 00000100:00100000:3.0:1789760728.438087:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4534 00000100:00100000:3.0:1789760728.438097:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709692425446 time 709727100523 state 0x2 ref 0 00000100:00100000:3.0:1789760728.441488:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.441614:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206144 00000100:00100000:1.0:1789760728.441625:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206144 00000100:00100000:1.0:1789760728.441681:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709692425446 time 709727100523 state 0x0 ref 1 00000100:00100000:1.0:1789760728.441738:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709727360939 time 709727100523 state 0x1 ref 1 00000100:00100000:1.0:1789760728.441763:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709727360939 time 709757066402 state 0x0 ref 1 00000100:00100000:1.0:1789760728.441777:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507206144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.441822:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18452480, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.441831:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.441845:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760728.441849:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.441855:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.441867:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.475069:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971825 is committed 00002000:00010000:0.0:1789760728.475166:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18452480 -> 18456576 00010000:00010000:0.0:1789760728.475182:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475195:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475204:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475286:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.475291:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475300:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475321:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.475330:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b776a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.475425:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507206144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33646us (33943us total) trans 4294971825 rc 0/0 00000100:00100000:0.0:1789760728.475435:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4535 00000100:00100000:0.0:1789760728.475444:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709727360939 time 709757066402 state 0x2 ref 0 00000100:00100000:2.0:1789760728.479465:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.479529:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206272 00000100:00100000:0.0:1789760728.479538:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206272 00000100:00100000:0.0:1789760728.479588:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709727360939 time 709757066402 state 0x0 ref 1 00000100:00100000:0.0:1789760728.479601:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709757326818 time 709757066402 state 0x1 ref 1 00000100:00100000:0.0:1789760728.479624:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709757326818 time 709794928265 state 0x0 ref 1 00000100:00100000:0.0:1789760728.479634:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507206272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.479678:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18456576, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.479686:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.479742:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.479747:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.479753:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.479767:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.522314:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971826 is committed 00002000:00010000:3.0:1789760728.522400:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18456576 -> 18460672 00010000:00010000:3.0:1789760728.522415:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522427:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522435:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522448:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760728.522452:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522461:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522481:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.522489:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7771 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.522615:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507206272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42976us (43198us total) trans 4294971826 rc 0/0 00000100:00100000:3.0:1789760728.522626:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4536 00000100:00100000:3.0:1789760728.522636:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709757326818 time 709794928265 state 0x2 ref 0 00000100:00100000:3.0:1789760728.531534:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.531642:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206400 00000100:00100000:0.0:1789760728.531651:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206400 00000100:00100000:0.0:1789760728.531739:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709757326818 time 709794928265 state 0x0 ref 1 00000100:00100000:0.0:1789760728.531756:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709795188681 time 709794928265 state 0x1 ref 1 00000100:00100000:0.0:1789760728.531777:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709795188681 time 709847081010 state 0x0 ref 1 00000100:00100000:0.0:1789760728.531791:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507206400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.531836:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18460672, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.531845:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.531862:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.531867:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.531873:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.531900:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.577909:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18460672 -> 18464768 00010000:00010000:1.0:1789760728.577937:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.577951:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.577959:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.577965:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.577970:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.577979:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.577999:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.578007:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7778 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.578100:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507206400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46306us (46570us total) trans 4294971827 rc 0/0 00000100:00100000:1.0:1789760728.578113:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4537 00000100:00100000:1.0:1789760728.578123:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709795188681 time 709847081010 state 0x2 ref 0 00000001:00080000:2.0:1789760728.578616:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971827 is committed 00000100:00100000:2.0:1789760728.585227:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.585455:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206528 00000100:00100000:0.0:1789760728.585466:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206528 00000100:00100000:0.0:1789760728.585525:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709795188681 time 709847081010 state 0x0 ref 1 00000100:00100000:0.0:1789760728.585542:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709847341426 time 709847081010 state 0x1 ref 1 00000100:00100000:0.0:1789760728.585573:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709847341426 time 709900876415 state 0x0 ref 1 00000100:00100000:0.0:1789760728.585585:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.585635:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18464768, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.585645:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.585660:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760728.585665:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.585671:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.585686:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.609882:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971828 is committed 00002000:00010000:0.0:1789760728.610064:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18464768 -> 18468864 00010000:00010000:0.0:1789760728.610080:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610093:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610104:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610111:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.610117:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610127:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610146:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.610154:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b777f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.610282:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24692us (25061us total) trans 4294971828 rc 0/0 00000100:00100000:0.0:1789760728.610292:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4538 00000100:00100000:0.0:1789760728.610303:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709847341426 time 709900876415 state 0x2 ref 0 00000100:00100000:3.0:1789760728.617804:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.618023:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206656 00000100:00100000:1.0:1789760728.618033:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206656 00000100:00100000:1.0:1789760728.618090:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709847341426 time 709900876415 state 0x0 ref 1 00000100:00100000:1.0:1789760728.618105:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709901136831 time 709900876415 state 0x1 ref 1 00000100:00100000:1.0:1789760728.618138:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709901136831 time 709933441066 state 0x0 ref 1 00000100:00100000:1.0:1789760728.618149:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.618198:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18468864, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.618208:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.618223:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760728.618227:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.618234:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.618247:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.647488:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971829 is committed 00002000:00010000:1.0:1789760728.647586:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18468864 -> 18472960 00010000:00010000:1.0:1789760728.647602:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647615:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647623:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647631:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.647636:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647645:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647665:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.647673:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7786 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.647802:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29650us (30004us total) trans 4294971829 rc 0/0 00000100:00100000:1.0:1789760728.647813:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4539 00000100:00100000:1.0:1789760728.647823:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709901136831 time 709933441066 state 0x2 ref 0 00000100:00100000:3.0:1789760728.651073:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.651225:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206784 00000100:00100000:1.0:1789760728.651235:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206784 00000100:00100000:1.0:1789760728.651290:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709901136831 time 709933441066 state 0x0 ref 1 00000100:00100000:1.0:1789760728.651306:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709933701482 time 709933441066 state 0x1 ref 1 00000100:00100000:1.0:1789760728.651344:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709933701482 time 709966646450 state 0x0 ref 1 00000100:00100000:1.0:1789760728.651356:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.651402:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18472960, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.651411:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.651426:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760728.651439:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.651446:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.651460:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.680042:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971830 is committed 00002000:00010000:0.0:1789760728.680246:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18472960 -> 18477056 00010000:00010000:0.0:1789760728.680261:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680272:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680280:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680292:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.680297:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680305:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680324:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.680403:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b778d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18472960->18477055] (req 18472960->18477055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.680539:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29179us (29470us total) trans 4294971830 rc 0/0 00000100:00100000:0.0:1789760728.680550:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4540 00000100:00100000:0.0:1789760728.680560:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709933701482 time 709966646450 state 0x2 ref 0 00000100:00100000:0.0:1789760728.689094:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.689186:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507206912 00000100:00100000:1.0:1789760728.689196:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507206912 00000100:00100000:1.0:1789760728.689248:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709933701482 time 709966646450 state 0x0 ref 1 00000100:00100000:1.0:1789760728.689264:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709966906866 time 709966646450 state 0x1 ref 1 00000100:00100000:1.0:1789760728.689298:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709966906866 time 710004601440 state 0x0 ref 1 00000100:00100000:1.0:1789760728.689311:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.689368:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18477056, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.689378:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.689392:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760728.689396:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.689401:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.689417:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.719796:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971831 is committed 00002000:00010000:0.0:1789760728.720038:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18477056 -> 18481152 00010000:00010000:0.0:1789760728.720056:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720070:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720161:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720169:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760728.720174:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720183:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720205:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.720214:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7794 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.720307:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507206912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30993us (31230us total) trans 4294971831 rc 0/0 00000100:00100000:0.0:1789760728.720388:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4541 00000100:00100000:0.0:1789760728.720400:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709966906866 time 710004601440 state 0x2 ref 0 00000100:00100000:2.0:1789760728.724496:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.724729:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207040 00000100:00100000:1.0:1789760728.724740:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207040 00000100:00100000:1.0:1789760728.724794:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 709966906866 time 710004601440 state 0x0 ref 1 00000100:00100000:1.0:1789760728.724810:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710004861856 time 710004601440 state 0x1 ref 1 00000100:00100000:1.0:1789760728.724900:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710004861856 time 710040202106 state 0x0 ref 1 00000100:00100000:1.0:1789760728.724913:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.724961:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18481152, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.724970:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.724984:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760728.724989:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.724996:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.725010:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.785941:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971832 is committed 00002000:00010000:1.0:1789760728.786091:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18481152 -> 18485248 00010000:00010000:1.0:1789760728.786106:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786118:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786125:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786132:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.786136:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786144:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786166:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.786173:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b779b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.786260:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61345us (61770us total) trans 4294971832 rc 0/0 00000100:00100000:1.0:1789760728.786269:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4542 00000100:00100000:1.0:1789760728.786278:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710004861856 time 710040202106 state 0x2 ref 0 00000100:00100000:3.0:1789760728.790904:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.791009:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207168 00000100:00100000:0.0:1789760728.791020:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207168 00000100:00100000:0.0:1789760728.791074:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710004861856 time 710040202106 state 0x0 ref 1 00000100:00100000:0.0:1789760728.791089:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710040462522 time 710040202106 state 0x1 ref 1 00000100:00100000:0.0:1789760728.791123:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710040462522 time 710106426418 state 0x0 ref 1 00000100:00100000:0.0:1789760728.791136:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507207168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.791277:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18485248, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.791286:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.791301:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760728.791305:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.791311:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.791388:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.855563:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18485248 -> 18489344 00010000:00010000:1.0:1789760728.855582:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855596:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855605:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855613:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.855618:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855633:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855652:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.855660:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.855778:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507207168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64641us (64895us total) trans 4294971833 rc 0/0 00000100:00100000:1.0:1789760728.855789:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4543 00000100:00100000:1.0:1789760728.855798:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710040462522 time 710106426418 state 0x2 ref 0 00000001:00080000:3.0:1789760728.855899:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971833 is committed 00000100:00100000:2.0:1789760728.862107:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760728.862290:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207296 00000100:00100000:0.0:1789760728.862299:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207296 00000100:00100000:0.0:1789760728.862355:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710040462522 time 710106426418 state 0x0 ref 1 00000100:00100000:0.0:1789760728.862369:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710106686834 time 710106426418 state 0x1 ref 1 00000100:00100000:0.0:1789760728.862398:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710106686834 time 710177701811 state 0x0 ref 1 00000100:00100000:0.0:1789760728.862409:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507207296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760728.862460:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18489344, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:0.0:1789760728.862469:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.862484:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760728.862488:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.862494:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.862508:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.923183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971834 is committed 00002000:00010000:1.0:1789760728.923367:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18489344 -> 18493440 00010000:00010000:1.0:1789760728.923553:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923569:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923578:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923584:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760728.923590:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923599:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923685:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.923798:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.924035:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507207296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61622us (61942us total) trans 4294971834 rc 0/0 00000100:00100000:1.0:1789760728.924046:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4544 00000100:00100000:1.0:1789760728.924056:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710106686834 time 710177701811 state 0x2 ref 0 00000100:00100000:2.0:1789760728.929845:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760728.929979:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207424 00000100:00100000:1.0:1789760728.929989:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207424 00000100:00100000:1.0:1789760728.930041:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710106686834 time 710177701811 state 0x0 ref 1 00000100:00100000:1.0:1789760728.930055:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710177962227 time 710177701811 state 0x1 ref 1 00000100:00100000:1.0:1789760728.930085:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710177962227 time 710245388236 state 0x0 ref 1 00000100:00100000:1.0:1789760728.930105:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760728.930152:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18493440, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760728.930171:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.930186:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760728.930192:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.930658:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.930679:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.988187:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760729.011579:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971835 is committed 00002000:00010000:0.0:1789760729.011840:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18493440 -> 18497536 00010000:00010000:0.0:1789760729.011855:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.011867:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.011876:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.011955:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.011959:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.011968:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.011993:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.012000:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b77b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.012615:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82512us (82772us total) trans 4294971835 rc 0/0 00000100:00100000:1.0:1789760729.012633:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4545 00000100:00100000:1.0:1789760729.012643:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710177962227 time 710245388236 state 0x2 ref 0 00000100:00100000:3.0:1789760729.020905:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760729.022867:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207552 00000100:00100000:1.0:1789760729.022889:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207552 00000100:00100000:1.0:1789760729.023142:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710177962227 time 710245388236 state 0x0 ref 1 00000100:00100000:1.0:1789760729.023157:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710245648652 time 710245388236 state 0x1 ref 1 00000100:00100000:1.0:1789760729.023188:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710245648652 time 710338491052 state 0x0 ref 1 00000100:00100000:1.0:1789760729.023202:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507207552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760729.023249:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18497536, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9007 00010000:00010000:1.0:1789760729.023258:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.023276:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760729.023280:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.023286:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.023299:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760729.030567:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760729.055953:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18497536 -> 18501632 00002000:00010000:2.0:1789760729.055964:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:2.0:1789760729.055967:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:2.0:1789760729.055979:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.055993:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.056003:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.056010:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.056016:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.056025:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.056045:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.056053:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b77b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.056148:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507207552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32944us (35262us total) trans 4294971836 rc 0/0 00000100:00100000:2.0:1789760729.056158:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4546 00000100:00100000:2.0:1789760729.056168:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710245648652 time 710338491052 state 0x2 ref 0 00000001:00080000:3.0:1789760729.056617:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971836 is committed 00000100:00100000:2.0:1789760729.059785:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760729.059872:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207680 00000100:00100000:1.0:1789760729.059890:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207680 00000100:00100000:1.0:1789760729.059952:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710245648652 time 710338491052 state 0x0 ref 1 00000100:00100000:1.0:1789760729.059967:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710338751468 time 710338491052 state 0x1 ref 1 00000100:00100000:1.0:1789760729.059997:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710338751468 time 710375300760 state 0x0 ref 1 00000100:00100000:1.0:1789760729.060009:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507207680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760729.060059:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18501632, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:1.0:1789760729.060068:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.060084:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760729.060090:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.060096:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.060114:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.095150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971837 is committed 00002000:00010000:1.0:1789760729.095269:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18501632 -> 18505728 00010000:00010000:1.0:1789760729.095284:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095296:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095305:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095313:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.095317:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095337:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095360:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095369:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b77be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.095464:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507207680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35452us (35684us total) trans 4294971837 rc 0/0 00000100:00100000:1.0:1789760729.095475:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4547 00000100:00100000:1.0:1789760729.095484:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710338751468 time 710375300760 state 0x2 ref 0 00000100:00100000:1.0:1789760729.104544:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.104659:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207808 00000100:00100000:2.0:1789760729.104670:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207808 00000100:00100000:2.0:1789760729.104758:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710338751468 time 710375300760 state 0x0 ref 1 00000100:00100000:2.0:1789760729.104773:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710375561176 time 710375300760 state 0x1 ref 1 00000100:00100000:2.0:1789760729.104800:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710375561176 time 710420104108 state 0x0 ref 1 00000100:00100000:2.0:1789760729.104813:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.104862:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18505728, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.104871:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.105347:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760729.105352:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.105359:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.105375:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.144610:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971838 is committed 00002000:00010000:0.0:1789760729.145458:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18505728 -> 18509824 00010000:00010000:0.0:1789760729.145474:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145486:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145495:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145502:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.145506:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145515:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145533:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.145545:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b77c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.145673:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507207808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40857us (41134us total) trans 4294971838 rc 0/0 00000100:00100000:0.0:1789760729.145683:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4548 00000100:00100000:0.0:1789760729.145724:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710375561176 time 710420104108 state 0x2 ref 0 00000100:00100000:0.0:1789760729.149251:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.149402:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507207936 00000100:00100000:2.0:1789760729.149412:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507207936 00000100:00100000:2.0:1789760729.149466:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710375561176 time 710420104108 state 0x0 ref 1 00000100:00100000:2.0:1789760729.149483:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710420364524 time 710420104108 state 0x1 ref 1 00000100:00100000:2.0:1789760729.149555:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710420364524 time 710464857908 state 0x0 ref 1 00000100:00100000:2.0:1789760729.149567:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507207936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.149613:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18509824, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.149622:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.149638:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760729.149642:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.149648:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.149662:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.177312:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971839 is committed 00002000:00010000:2.0:1789760729.177452:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18509824 -> 18513920 00010000:00010000:2.0:1789760729.177468:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177482:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177492:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177505:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.177510:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177519:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177538:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.177546:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.177670:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507207936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28100us (28424us total) trans 4294971839 rc 0/0 00000100:00100000:2.0:1789760729.177679:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4549 00000100:00100000:2.0:1789760729.177689:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710420364524 time 710464857908 state 0x2 ref 0 00000100:00100000:0.0:1789760729.182128:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760729.182226:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208064 00000100:00100000:1.0:1789760729.182235:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208064 00000100:00100000:1.0:1789760729.182290:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710420364524 time 710464857908 state 0x0 ref 1 00000100:00100000:1.0:1789760729.182304:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710465118324 time 710464857908 state 0x1 ref 1 00000100:00100000:1.0:1789760729.182427:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710465118324 time 710497728621 state 0x0 ref 1 00000100:00100000:1.0:1789760729.182441:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760729.182490:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18513920, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:1.0:1789760729.182501:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.182517:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760729.182522:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.182529:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.182543:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.207525:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971840 is committed 00002000:00010000:1.0:1789760729.207690:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18513920 -> 18518016 00010000:00010000:1.0:1789760729.207732:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207744:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207753:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207761:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.207765:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207774:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207794:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.207802:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b77d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.207877:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25435us (25753us total) trans 4294971840 rc 0/0 00000100:00100000:1.0:1789760729.207902:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4550 00000100:00100000:1.0:1789760729.207912:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710465118324 time 710497728621 state 0x2 ref 0 00000100:00100000:1.0:1789760729.211108:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.211197:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208192 00000100:00100000:0.0:1789760729.211206:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208192 00000100:00100000:0.0:1789760729.211259:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710465118324 time 710497728621 state 0x0 ref 1 00000100:00100000:0.0:1789760729.211274:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710497989037 time 710497728621 state 0x1 ref 1 00000100:00100000:0.0:1789760729.211337:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710497989037 time 710526640263 state 0x0 ref 1 00000100:00100000:0.0:1789760729.211350:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.211397:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18518016, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.211406:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.211422:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.211427:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.211434:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.211448:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.234159:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971841 is committed 00002000:00010000:0.0:1789760729.234322:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18518016 -> 18522112 00010000:00010000:0.0:1789760729.234405:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234417:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234425:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234432:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.234437:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234445:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234465:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.234472:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.234564:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23211us (23462us total) trans 4294971841 rc 0/0 00000100:00100000:0.0:1789760729.234575:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4551 00000100:00100000:0.0:1789760729.234584:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710497989037 time 710526640263 state 0x2 ref 0 00000100:00100000:0.0:1789760729.238239:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.238462:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208320 00000100:00100000:2.0:1789760729.238472:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208320 00000100:00100000:2.0:1789760729.238527:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710497989037 time 710526640263 state 0x0 ref 1 00000100:00100000:2.0:1789760729.238543:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710526900679 time 710526640263 state 0x1 ref 1 00000100:00100000:2.0:1789760729.238573:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710526900679 time 710553876645 state 0x0 ref 1 00000100:00100000:2.0:1789760729.238585:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507208320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.238633:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18522112, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.238642:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.238657:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760729.238661:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.238667:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.238681:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.268270:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971842 is committed 00002000:00010000:2.0:1789760729.268503:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18522112 -> 18526208 00010000:00010000:2.0:1789760729.268518:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268530:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268538:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268545:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.268550:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268560:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268580:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.268587:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.268675:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507208320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30084us (30438us total) trans 4294971842 rc 0/0 00000100:00100000:2.0:1789760729.268685:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4552 00000100:00100000:2.0:1789760729.268723:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710526900679 time 710553876645 state 0x2 ref 0 00000100:00100000:1.0:1789760729.271158:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.271236:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208448 00000100:00100000:2.0:1789760729.271245:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208448 00000100:00100000:2.0:1789760729.271298:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710526900679 time 710553876645 state 0x0 ref 1 00000100:00100000:2.0:1789760729.271488:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710554137061 time 710553876645 state 0x1 ref 1 00000100:00100000:2.0:1789760729.271522:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710554137061 time 710586824581 state 0x0 ref 1 00000100:00100000:2.0:1789760729.271535:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.271580:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18526208, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.271588:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271602:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760729.271606:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.271612:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271625:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.307478:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971843 is committed 00002000:00010000:0.0:1789760729.307602:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18526208 -> 18530304 00010000:00010000:0.0:1789760729.307617:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307638:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307647:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307654:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.307658:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307667:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307685:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.307729:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b77e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.307811:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36274us (36658us total) trans 4294971843 rc 0/0 00000100:00100000:0.0:1789760729.307821:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4553 00000100:00100000:0.0:1789760729.307831:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710554137061 time 710586824581 state 0x2 ref 0 00000100:00100000:1.0:1789760729.309085:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22782b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538268544:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760729.309140:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760729.310396:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268544 00000100:00100000:1.0:1789760729.310416:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268544 00000100:00100000:1.0:1789760729.310469:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:1.0:1789760729.310478:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538268544:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760729.310505:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3839885312 avail 3824156672 00000100:00100000:1.0:1789760729.310568:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538268544:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (1427us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760729.310574:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:2.0:1789760729.310776:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22782b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538268544:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760729.311122:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538268672:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760729.311157:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760729.311257:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268672 00000100:00100000:1.0:1789760729.311261:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268672 00000100:00100000:1.0:1789760729.311297:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:1.0:1789760729.311304:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538268672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760729.311316:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3839885312 avail 3824156672 00000100:00100000:1.0:1789760729.311650:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538268672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 346us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760729.311658:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:0.0:1789760729.311765:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538268672:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760729.313168:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.313545:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208576 00000100:00100000:0.0:1789760729.313552:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208576 00000100:00100000:0.0:1789760729.313599:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710554137061 time 710586824581 state 0x0 ref 1 00000100:00100000:0.0:1789760729.313612:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710587084997 time 710586824581 state 0x1 ref 1 00000100:00100000:0.0:1789760729.313680:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710587084997 time 710628979719 state 0x0 ref 1 00000100:00100000:0.0:1789760729.313707:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507208576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.313749:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18530304, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.313757:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.313773:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.313776:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.313781:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.313790:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.342810:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971844 is committed 00002000:00010000:3.0:1789760729.342935:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18530304 -> 18534400 00010000:00010000:3.0:1789760729.342950:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.342962:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.342971:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.342978:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760729.342982:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.342992:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.343010:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.343023:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b77ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.343110:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507208576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29415us (29943us total) trans 4294971844 rc 0/0 00000100:00100000:3.0:1789760729.343119:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4554 00000100:00100000:3.0:1789760729.343128:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710587084997 time 710628979719 state 0x2 ref 0 00000100:00100000:3.0:1789760729.346485:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.346747:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208704 00000100:00100000:2.0:1789760729.346756:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208704 00000100:00100000:2.0:1789760729.346810:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710587084997 time 710628979719 state 0x0 ref 1 00000100:00100000:2.0:1789760729.346824:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710629240135 time 710628979719 state 0x1 ref 1 00000100:00100000:2.0:1789760729.346854:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710629240135 time 710662157112 state 0x0 ref 1 00000100:00100000:2.0:1789760729.346866:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.346922:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18534400, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.346931:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.346946:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760729.346951:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.346957:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.346972:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.416198:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971845 is committed 00002000:00010000:1.0:1789760729.416443:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18534400 -> 18538496 00010000:00010000:1.0:1789760729.416458:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416471:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416480:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416487:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.416491:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416501:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416518:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.416526:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b77f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.416616:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69747us (70135us total) trans 4294971845 rc 0/0 00000100:00100000:1.0:1789760729.416625:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4555 00000100:00100000:1.0:1789760729.416635:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710629240135 time 710662157112 state 0x2 ref 0 00000100:00100000:1.0:1789760729.423125:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.423214:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208832 00000100:00100000:0.0:1789760729.423232:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208832 00000100:00100000:0.0:1789760729.423284:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710629240135 time 710662157112 state 0x0 ref 1 00000100:00100000:0.0:1789760729.423299:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710662417528 time 710662157112 state 0x1 ref 1 00000100:00100000:0.0:1789760729.423421:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710662417528 time 710738723321 state 0x0 ref 1 00000100:00100000:0.0:1789760729.423435:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.423482:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18538496, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.423492:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.423507:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.423512:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.423527:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.423546:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.461793:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971846 is committed 00002000:00010000:1.0:1789760729.461926:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18538496 -> 18542592 00010000:00010000:1.0:1789760729.461941:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.461957:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.461966:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.461976:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.461980:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.461990:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.462009:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.462016:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b77fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.462107:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507208832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38668us (38984us total) trans 4294971846 rc 0/0 00000100:00100000:1.0:1789760729.462120:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4556 00000100:00100000:1.0:1789760729.462129:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710662417528 time 710738723321 state 0x2 ref 0 00000100:00100000:2.0:1789760729.467436:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760729.469232:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507208960 00000100:00100000:3.0:1789760729.469245:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507208960 00000100:00100000:3.0:1789760729.469307:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710662417528 time 710738723321 state 0x0 ref 1 00000100:00100000:3.0:1789760729.469321:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710738983737 time 710738723321 state 0x1 ref 1 00000100:00100000:3.0:1789760729.469356:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710738983737 time 710784658963 state 0x0 ref 1 00000100:00100000:3.0:1789760729.469369:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760729.469415:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18542592, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:3.0:1789760729.469424:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.469439:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760729.469443:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.469449:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.469463:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.519679:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971847 is committed 00002000:00010000:3.0:1789760729.519841:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18542592 -> 18546688 00010000:00010000:3.0:1789760729.519855:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519867:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519886:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519893:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760729.519898:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519906:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519974:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.519982:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7804 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.520072:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507208960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50701us (52704us total) trans 4294971847 rc 0/0 00000100:00100000:3.0:1789760729.520082:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4557 00000100:00100000:3.0:1789760729.520091:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710738983737 time 710784658963 state 0x2 ref 0 00000100:00100000:3.0:1789760729.523232:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760729.523638:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209088 00000100:00100000:1.0:1789760729.523647:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209088 00000100:00100000:1.0:1789760729.523788:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710738983737 time 710784658963 state 0x0 ref 1 00000100:00100000:1.0:1789760729.523805:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710784919379 time 710784658963 state 0x1 ref 1 00000100:00100000:1.0:1789760729.523830:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710784919379 time 710839134425 state 0x0 ref 1 00000100:00100000:1.0:1789760729.523840:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507209088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760729.523891:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18546688, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:1.0:1789760729.523899:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.524083:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760729.524089:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.524095:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.524109:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.551807:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971848 is committed 00002000:00010000:0.0:1789760729.551989:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18546688 -> 18550784 00010000:00010000:0.0:1789760729.552006:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552019:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552028:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552036:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.552041:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552050:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552069:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.552076:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b780b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.552204:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507209088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28360us (28975us total) trans 4294971848 rc 0/0 00000100:00100000:0.0:1789760729.552215:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4558 00000100:00100000:0.0:1789760729.552225:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710784919379 time 710839134425 state 0x2 ref 0 00000100:00100000:1.0:1789760729.556762:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.556851:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209216 00000100:00100000:0.0:1789760729.556861:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209216 00000100:00100000:0.0:1789760729.556923:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710784919379 time 710839134425 state 0x0 ref 1 00000100:00100000:0.0:1789760729.556937:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710839394841 time 710839134425 state 0x1 ref 1 00000100:00100000:0.0:1789760729.556959:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710839394841 time 710872262362 state 0x0 ref 1 00000100:00100000:0.0:1789760729.556971:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507209216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.557020:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18550784, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.557029:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.557044:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.557048:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.557054:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.557067:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.581316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971849 is committed 00002000:00010000:0.0:1789760729.581539:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18550784 -> 18554880 00010000:00010000:0.0:1789760729.581554:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581568:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581579:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581586:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.581591:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581600:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581618:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.581626:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7812 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.581738:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507209216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24765us (24981us total) trans 4294971849 rc 0/0 00000100:00100000:0.0:1789760729.581751:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4559 00000100:00100000:0.0:1789760729.581760:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710839394841 time 710872262362 state 0x2 ref 0 00000100:00100000:2.0:1789760729.585819:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760729.585991:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209344 00000100:00100000:3.0:1789760729.586001:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209344 00000100:00100000:3.0:1789760729.586056:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710839394841 time 710872262362 state 0x0 ref 1 00000100:00100000:3.0:1789760729.586071:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710872522778 time 710872262362 state 0x1 ref 1 00000100:00100000:3.0:1789760729.586097:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710872522778 time 710901400851 state 0x0 ref 1 00000100:00100000:3.0:1789760729.586110:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507209344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760729.586227:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18554880, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:3.0:1789760729.586236:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.586250:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760729.586254:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.586260:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.586273:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.613732:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18554880 -> 18558976 00010000:00010000:3.0:1789760729.613751:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613763:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613772:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613779:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760729.613784:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613793:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613811:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.613818:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7819 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.613922:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507209344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27803us (28100us total) trans 4294971850 rc 0/0 00000100:00100000:3.0:1789760729.613934:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4560 00000100:00100000:3.0:1789760729.613943:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710872522778 time 710901400851 state 0x2 ref 0 00000001:00080000:0.0:1789760729.614519:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971850 is committed 00000100:00100000:3.0:1789760729.617145:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.617233:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209472 00000100:00100000:0.0:1789760729.617243:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209472 00000100:00100000:0.0:1789760729.617295:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710872522778 time 710901400851 state 0x0 ref 1 00000100:00100000:0.0:1789760729.617309:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710901661267 time 710901400851 state 0x1 ref 1 00000100:00100000:0.0:1789760729.617381:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710901661267 time 710932684462 state 0x0 ref 1 00000100:00100000:0.0:1789760729.617394:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.617440:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18558976, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.617449:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.617466:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.617471:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.617477:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.617490:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.650127:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971851 is committed 00002000:00010000:0.0:1789760729.650203:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18558976 -> 18563072 00010000:00010000:0.0:1789760729.650218:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650231:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650240:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650246:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.650251:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650260:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650278:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.650285:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7820 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.650405:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33008us (33264us total) trans 4294971851 rc 0/0 00000100:00100000:0.0:1789760729.650414:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4561 00000100:00100000:0.0:1789760729.650423:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710901661267 time 710932684462 state 0x2 ref 0 00000100:00100000:1.0:1789760729.667179:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.667815:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209600 00000100:00100000:0.0:1789760729.667825:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209600 00000100:00100000:0.0:1789760729.667890:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710901661267 time 710932684462 state 0x0 ref 1 00000100:00100000:0.0:1789760729.667906:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710932944878 time 710932684462 state 0x1 ref 1 00000100:00100000:0.0:1789760729.667940:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710932944878 time 710983243630 state 0x0 ref 1 00000100:00100000:0.0:1789760729.667952:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.668000:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18563072, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.668009:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.668026:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760729.668030:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.668037:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.668050:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.710434:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971852 is committed 00002000:00010000:0.0:1789760729.712994:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18563072 -> 18567168 00010000:00010000:0.0:1789760729.713013:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713025:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713034:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713042:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760729.713047:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713056:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713075:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.713084:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7827 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.713181:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45225us (46008us total) trans 4294971852 rc 0/0 00000100:00100000:0.0:1789760729.713191:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4562 00000100:00100000:0.0:1789760729.713201:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710932944878 time 710983243630 state 0x2 ref 0 00000100:00100000:2.0:1789760729.719339:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760729.719411:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209728 00000100:00100000:3.0:1789760729.719421:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209728 00000100:00100000:3.0:1789760729.719472:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710932944878 time 710983243630 state 0x0 ref 1 00000100:00100000:3.0:1789760729.719486:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710983504046 time 710983243630 state 0x1 ref 1 00000100:00100000:3.0:1789760729.719515:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710983504046 time 711034818732 state 0x0 ref 1 00000100:00100000:3.0:1789760729.719528:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760729.719574:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18567168, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:3.0:1789760729.719584:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.719598:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760729.719602:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.719608:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.719620:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.747972:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18567168 -> 18571264 00010000:00010000:3.0:1789760729.747990:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748001:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748011:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748018:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760729.748023:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748031:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748049:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.748056:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b782e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.748143:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507209728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28612us (28819us total) trans 4294971853 rc 0/0 00000100:00100000:3.0:1789760729.748155:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4563 00000100:00100000:3.0:1789760729.748163:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710983504046 time 711034818732 state 0x2 ref 0 00000001:00080000:3.0:1789760729.748198:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971853 is committed 00000100:00100000:3.0:1789760729.751462:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.751737:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209856 00000100:00100000:0.0:1789760729.751746:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209856 00000100:00100000:0.0:1789760729.751796:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 710983504046 time 711034818732 state 0x0 ref 1 00000100:00100000:0.0:1789760729.751809:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711035079148 time 711034818732 state 0x1 ref 1 00000100:00100000:0.0:1789760729.751832:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711035079148 time 711067135911 state 0x0 ref 1 00000100:00100000:0.0:1789760729.751842:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507209856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.751894:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18571264, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.751902:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.751923:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760729.751927:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.751933:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.751946:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.785283:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971854 is committed 00002000:00010000:2.0:1789760729.785344:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18571264 -> 18575360 00010000:00010000:2.0:1789760729.785359:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785372:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785382:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785390:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.785399:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785409:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785433:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.785442:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7835 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.785602:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507209856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33756us (34143us total) trans 4294971854 rc 0/0 00000100:00100000:2.0:1789760729.785612:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4564 00000100:00100000:2.0:1789760729.785621:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711035079148 time 711067135911 state 0x2 ref 0 00000100:00100000:0.0:1789760729.788623:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760729.788761:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507209984 00000100:00100000:1.0:1789760729.788771:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507209984 00000100:00100000:1.0:1789760729.788823:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711035079148 time 711067135911 state 0x0 ref 1 00000100:00100000:1.0:1789760729.788838:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711067396327 time 711067135911 state 0x1 ref 1 00000100:00100000:1.0:1789760729.788911:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711067396327 time 711104213141 state 0x0 ref 1 00000100:00100000:1.0:1789760729.788924:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760729.788970:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18575360, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:1.0:1789760729.788978:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.788993:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760729.788997:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.789004:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.789017:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.824260:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971855 is committed 00002000:00010000:2.0:1789760729.824778:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18575360 -> 18579456 00010000:00010000:2.0:1789760729.824793:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824807:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824816:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824825:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.824829:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824840:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824865:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.824872:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b783c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.824983:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507209984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36057us (36364us total) trans 4294971855 rc 0/0 00000100:00100000:2.0:1789760729.824993:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4565 00000100:00100000:2.0:1789760729.825003:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711067396327 time 711104213141 state 0x2 ref 0 00000100:00100000:2.0:1789760729.827439:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227829c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538268800:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760729.827508:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760729.827919:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268800 00000100:00100000:3.0:1789760729.827927:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268800 00000100:00100000:3.0:1789760729.827978:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1789760729.827988:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538268800:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760729.828025:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760729.828123:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538268800:12345-0@lo:41:osp-pre-1-0.0 Request processed in 136us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760729.828130:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:2.0:1789760729.828172:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227829c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538268800:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760729.829002:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760729.829233:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210112 00000100:00100000:3.0:1789760729.829239:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210112 00000100:00100000:3.0:1789760729.829281:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711067396327 time 711104213141 state 0x0 ref 1 00000100:00100000:3.0:1789760729.829294:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711104473557 time 711104213141 state 0x1 ref 1 00000100:00100000:3.0:1789760729.829374:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711104473557 time 711144626563 state 0x0 ref 1 00000100:00100000:3.0:1789760729.829387:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507210112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760729.829426:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18579456, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:3.0:1789760729.829435:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.829450:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760729.829455:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.829461:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.829475:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.862953:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18579456 -> 18583552 00010000:00010000:1.0:1789760729.862972:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862989:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862998:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.863006:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.863010:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.863019:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.863038:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.863045:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7843 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.863132:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507210112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33742us (34129us total) trans 4294971856 rc 0/0 00000100:00100000:1.0:1789760729.863142:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4566 00000100:00100000:1.0:1789760729.863161:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711104473557 time 711144626563 state 0x2 ref 0 00000001:00080000:2.0:1789760729.863526:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971856 is committed 00000100:00100000:3.0:1789760729.869776:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.869965:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210240 00000100:00100000:2.0:1789760729.869975:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210240 00000100:00100000:2.0:1789760729.870028:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711104473557 time 711144626563 state 0x0 ref 1 00000100:00100000:2.0:1789760729.870042:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711144886979 time 711144626563 state 0x1 ref 1 00000100:00100000:2.0:1789760729.870075:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711144886979 time 711185378284 state 0x0 ref 1 00000100:00100000:2.0:1789760729.870087:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507210240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.870139:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18583552, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.870158:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.870174:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760729.870179:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.870185:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.870200:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.922149:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971857 is committed 00002000:00010000:2.0:1789760729.922248:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18583552 -> 18587648 00010000:00010000:2.0:1789760729.922263:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922274:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922282:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922288:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760729.922292:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922300:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922390:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.922398:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b784a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.922489:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507210240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52400us (52718us total) trans 4294971857 rc 0/0 00000100:00100000:2.0:1789760729.922499:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4567 00000100:00100000:2.0:1789760729.922508:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711144886979 time 711185378284 state 0x2 ref 0 00000100:00100000:0.0:1789760729.929602:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760729.929817:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210368 00000100:00100000:2.0:1789760729.929827:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210368 00000100:00100000:2.0:1789760729.929891:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711144886979 time 711185378284 state 0x0 ref 1 00000100:00100000:2.0:1789760729.929907:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711185638700 time 711185378284 state 0x1 ref 1 00000100:00100000:2.0:1789760729.929937:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711185638700 time 711245240804 state 0x0 ref 1 00000100:00100000:2.0:1789760729.929951:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760729.929999:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18587648, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:2.0:1789760729.930008:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.930023:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760729.930028:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.930034:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.930046:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.959869:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18587648 -> 18591744 00010000:00010000:1.0:1789760729.959900:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959923:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959932:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959939:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760729.959943:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959952:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959971:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.959978:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7851 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.960072:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507210368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30118us (30486us total) trans 4294971858 rc 0/0 00000100:00100000:1.0:1789760729.960083:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4568 00000100:00100000:1.0:1789760729.960092:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711185638700 time 711245240804 state 0x2 ref 0 00000001:00080000:3.0:1789760729.960362:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971858 is committed 00000100:00100000:1.0:1789760729.964144:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760729.964256:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210496 00000100:00100000:3.0:1789760729.964266:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210496 00000100:00100000:3.0:1789760729.964335:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711185638700 time 711245240804 state 0x0 ref 1 00000100:00100000:3.0:1789760729.964351:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711245501220 time 711245240804 state 0x1 ref 1 00000100:00100000:3.0:1789760729.964380:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711245501220 time 711279683767 state 0x0 ref 1 00000100:00100000:3.0:1789760729.964393:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760729.964441:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18591744, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:3.0:1789760729.964450:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.964465:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760729.964469:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.964476:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.964490:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.993159:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971859 is committed 00002000:00010000:3.0:1789760729.993225:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18591744 -> 18595840 00010000:00010000:3.0:1789760729.993238:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993249:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993256:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993263:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760729.993267:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993276:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993293:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.993300:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7858 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.993505:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29109us (29366us total) trans 4294971859 rc 0/0 00000100:00100000:3.0:1789760729.993515:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4569 00000100:00100000:3.0:1789760729.993524:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711245501220 time 711279683767 state 0x2 ref 0 00000100:00100000:3.0:1789760729.996772:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760729.997036:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210624 00000100:00100000:0.0:1789760729.997046:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210624 00000100:00100000:0.0:1789760729.997100:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711245501220 time 711279683767 state 0x0 ref 1 00000100:00100000:0.0:1789760729.997115:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711279944183 time 711279683767 state 0x1 ref 1 00000100:00100000:0.0:1789760729.997146:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711279944183 time 711312449162 state 0x0 ref 1 00000100:00100000:0.0:1789760729.997159:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760729.997205:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18595840, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9007 00010000:00010000:0.0:1789760729.997213:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.997229:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760729.997234:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.997240:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.997254:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.011790:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760730.027346:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971860 is committed 00002000:00010000:2.0:1789760730.027536:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18595840 -> 18599936 00002000:00010000:2.0:1789760730.027541:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:2.0:1789760730.027544:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:2.0:1789760730.027557:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027569:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027578:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027585:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.027589:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027598:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027618:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.027627:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b785f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.027746:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30585us (30979us total) trans 4294971860 rc 0/0 00000100:00100000:2.0:1789760730.027756:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4570 00000100:00100000:2.0:1789760730.027764:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711279944183 time 711312449162 state 0x2 ref 0 00000100:00100000:3.0:1789760730.031196:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.031544:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210752 00000100:00100000:2.0:1789760730.031553:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210752 00000100:00100000:2.0:1789760730.031601:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711279944183 time 711312449162 state 0x0 ref 1 00000100:00100000:2.0:1789760730.031615:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711312709578 time 711312449162 state 0x1 ref 1 00000100:00100000:2.0:1789760730.031645:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711312709578 time 711346948628 state 0x0 ref 1 00000100:00100000:2.0:1789760730.031656:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.031718:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18599936, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.031728:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.031742:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.031746:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.031753:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.031772:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.064592:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18599936 -> 18604032 00010000:00010000:2.0:1789760730.064610:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064623:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064633:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064640:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.064646:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064655:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064675:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.064683:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7866 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.064804:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507210752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33143us (33610us total) trans 4294971861 rc 0/0 00000100:00100000:2.0:1789760730.064814:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4571 00000100:00100000:2.0:1789760730.064824:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711312709578 time 711346948628 state 0x2 ref 0 00000001:00080000:3.0:1789760730.067752:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971861 is committed 00000100:00100000:1.0:1789760730.067988:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760730.068098:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507210880 00000100:00100000:3.0:1789760730.068107:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507210880 00000100:00100000:3.0:1789760730.068156:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711312709578 time 711346948628 state 0x0 ref 1 00000100:00100000:3.0:1789760730.068170:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711347209044 time 711346948628 state 0x1 ref 1 00000100:00100000:3.0:1789760730.068569:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711347209044 time 711383871786 state 0x0 ref 1 00000100:00100000:3.0:1789760730.068583:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760730.068629:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18604032, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:3.0:1789760730.068638:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.068654:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760730.068659:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.068666:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.068680:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.096685:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971862 is committed 00002000:00010000:1.0:1789760730.096848:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18604032 -> 18608128 00010000:00010000:1.0:1789760730.096863:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096886:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096896:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096903:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760730.096917:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096926:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096945:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.096952:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b786d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.097043:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507210880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28457us (29053us total) trans 4294971862 rc 0/0 00000100:00100000:1.0:1789760730.097053:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4572 00000100:00100000:1.0:1789760730.097063:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711347209044 time 711383871786 state 0x2 ref 0 00000100:00100000:0.0:1789760730.102618:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.102772:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211008 00000100:00100000:2.0:1789760730.102781:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211008 00000100:00100000:2.0:1789760730.102833:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711347209044 time 711383871786 state 0x0 ref 1 00000100:00100000:2.0:1789760730.102848:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711384132202 time 711383871786 state 0x1 ref 1 00000100:00100000:2.0:1789760730.103004:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711384132202 time 711418306022 state 0x0 ref 1 00000100:00100000:2.0:1789760730.103016:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507211008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.103067:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18608128, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.103076:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.103092:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.103096:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.103103:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.103117:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.128031:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18608128 -> 18612224 00010000:00010000:3.0:1789760730.128050:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128063:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128072:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128080:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.128085:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128094:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128114:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.128122:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7874 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.128211:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507211008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25192us (25597us total) trans 4294971863 rc 0/0 00000100:00100000:3.0:1789760730.128221:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4573 00000100:00100000:3.0:1789760730.128231:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711384132202 time 711418306022 state 0x2 ref 0 00000001:00080000:3.0:1789760730.128268:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971863 is committed 00000100:00100000:3.0:1789760730.134441:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.134531:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211136 00000100:00100000:2.0:1789760730.134541:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211136 00000100:00100000:2.0:1789760730.134593:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711384132202 time 711418306022 state 0x0 ref 1 00000100:00100000:2.0:1789760730.134609:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711418566438 time 711418306022 state 0x1 ref 1 00000100:00100000:2.0:1789760730.134637:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711418566438 time 711449941050 state 0x0 ref 1 00000100:00100000:2.0:1789760730.134649:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507211136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.134725:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18612224, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.134733:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.134747:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.134751:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.134758:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.134770:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.163499:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18612224 -> 18616320 00010000:00010000:1.0:1789760730.163518:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163531:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163541:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163548:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760730.163553:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163562:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163582:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.163589:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b787b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.163683:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507211136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29030us (29246us total) trans 4294971864 rc 0/0 00000100:00100000:1.0:1789760730.163724:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4574 00000100:00100000:1.0:1789760730.163735:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711418566438 time 711449941050 state 0x2 ref 0 00000001:00080000:3.0:1789760730.163822:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971864 is committed 00000100:00100000:1.0:1789760730.167448:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.167810:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211264 00000100:00100000:2.0:1789760730.167975:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211264 00000100:00100000:2.0:1789760730.168032:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711418566438 time 711449941050 state 0x0 ref 1 00000100:00100000:2.0:1789760730.168048:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711450201466 time 711449941050 state 0x1 ref 1 00000100:00100000:2.0:1789760730.168122:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711450201466 time 711483424551 state 0x0 ref 1 00000100:00100000:2.0:1789760730.168135:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.168183:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18616320, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.168192:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.168206:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.168211:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.168218:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.168233:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.205496:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971865 is committed 00002000:00010000:1.0:1789760730.207751:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18616320 -> 18620416 00010000:00010000:1.0:1789760730.207770:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207782:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207792:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207799:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760730.207803:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207813:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207834:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.207845:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7882 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.208633:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40493us (41188us total) trans 4294971865 rc 0/0 00000100:00100000:1.0:1789760730.208646:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4575 00000100:00100000:1.0:1789760730.208656:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711450201466 time 711483424551 state 0x2 ref 0 00000100:00100000:0.0:1789760730.211689:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760730.211783:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211392 00000100:00100000:3.0:1789760730.211792:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211392 00000100:00100000:3.0:1789760730.211846:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711450201466 time 711483424551 state 0x0 ref 1 00000100:00100000:3.0:1789760730.211860:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711483684967 time 711483424551 state 0x1 ref 1 00000100:00100000:3.0:1789760730.211890:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711483684967 time 711527193932 state 0x0 ref 1 00000100:00100000:3.0:1789760730.211902:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760730.212028:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18620416, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:3.0:1789760730.212037:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.212052:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760730.212056:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.212063:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.212077:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.259682:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971866 is committed 00002000:00010000:3.0:1789760730.259808:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18620416 -> 18624512 00010000:00010000:3.0:1789760730.259823:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259836:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259845:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259852:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.259857:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259867:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259897:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.259905:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7889 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.260090:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48184us (48405us total) trans 4294971866 rc 0/0 00000100:00100000:3.0:1789760730.260099:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4576 00000100:00100000:3.0:1789760730.260109:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711483684967 time 711527193932 state 0x2 ref 0 00000100:00100000:3.0:1789760730.269132:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760730.271292:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211520 00000100:00100000:1.0:1789760730.271306:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211520 00000100:00100000:0.0:1789760730.271519:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711483684967 time 711527193932 state 0x0 ref 1 00000100:00100000:0.0:1789760730.271535:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711527454348 time 711527193932 state 0x1 ref 1 00000100:00100000:0.0:1789760730.271609:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711527454348 time 711586911750 state 0x0 ref 1 00000100:00100000:0.0:1789760730.271621:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507211520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760730.271668:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18624512, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:0.0:1789760730.271677:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.271715:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760730.271721:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.271727:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.271741:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.313209:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18624512 -> 18628608 00010000:00010000:0.0:1789760730.313296:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313309:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313317:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313324:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760730.313329:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313338:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313357:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.313371:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7890 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.313536:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507211520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41912us (44409us total) trans 4294971867 rc 0/0 00000100:00100000:0.0:1789760730.313548:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4577 00000100:00100000:0.0:1789760730.313559:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711527454348 time 711586911750 state 0x2 ref 0 00000001:00080000:1.0:1789760730.314308:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971867 is committed 00000100:00100000:1.0:1789760730.318621:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.318734:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211648 00000100:00100000:2.0:1789760730.318743:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211648 00000100:00100000:2.0:1789760730.318795:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711527454348 time 711586911750 state 0x0 ref 1 00000100:00100000:2.0:1789760730.318823:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711587172166 time 711586911750 state 0x1 ref 1 00000100:00100000:2.0:1789760730.318852:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711587172166 time 711634155347 state 0x0 ref 1 00000100:00100000:2.0:1789760730.318866:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507211648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.319092:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18628608, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.319101:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.319116:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.319120:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.319126:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.319138:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.332091:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227828000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538268928:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760730.332167:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760730.332274:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538268928 00000100:00100000:3.0:1789760730.332282:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538268928 00000100:00100000:3.0:1789760730.332331:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1789760730.332341:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538268928:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760730.332377:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760730.335153:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538268928:12345-0@lo:41:osp-pre-0-1.0 Request processed in 2806us (2978us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760730.335166:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:1.0:1789760730.335668:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227828000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538268928:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760730.362844:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971868 is committed 00002000:00010000:2.0:1789760730.363090:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18628608 -> 18632704 00010000:00010000:2.0:1789760730.363106:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363119:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363128:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363135:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.363141:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363151:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363175:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.363182:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7897 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.363311:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507211648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44443us (44697us total) trans 4294971868 rc 0/0 00000100:00100000:2.0:1789760730.363322:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4578 00000100:00100000:2.0:1789760730.363332:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711587172166 time 711634155347 state 0x2 ref 0 00000100:00100000:3.0:1789760730.366966:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.367112:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211776 00000100:00100000:2.0:1789760730.367122:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211776 00000100:00100000:2.0:1789760730.367175:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711587172166 time 711634155347 state 0x0 ref 1 00000100:00100000:2.0:1789760730.367190:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711634415763 time 711634155347 state 0x1 ref 1 00000100:00100000:2.0:1789760730.367219:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711634415763 time 711682522527 state 0x0 ref 1 00000100:00100000:2.0:1789760730.367232:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.367282:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18632704, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.367290:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.367322:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.367327:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.367333:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.367346:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.405979:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971869 is committed 00002000:00010000:2.0:1789760730.406455:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18632704 -> 18636800 00010000:00010000:2.0:1789760730.406471:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406483:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406493:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406501:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.406506:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406515:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406534:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.406541:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b789e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.406663:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507211776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39429us (39703us total) trans 4294971869 rc 0/0 00000100:00100000:2.0:1789760730.406674:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4579 00000100:00100000:2.0:1789760730.406684:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711634415763 time 711682522527 state 0x2 ref 0 00000100:00100000:3.0:1789760730.412192:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760730.412277:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507211904 00000100:00100000:0.0:1789760730.412286:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507211904 00000100:00100000:0.0:1789760730.412345:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711634415763 time 711682522527 state 0x0 ref 1 00000100:00100000:0.0:1789760730.412360:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711682782943 time 711682522527 state 0x1 ref 1 00000100:00100000:0.0:1789760730.412387:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711682782943 time 711727690064 state 0x0 ref 1 00000100:00100000:0.0:1789760730.412398:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507211904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760730.412452:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18636800, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:0.0:1789760730.412461:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.412477:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760730.412481:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.412488:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.412507:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.438545:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18636800 -> 18640896 00010000:00010000:3.0:1789760730.438565:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438579:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438588:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438595:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.438600:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438608:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438628:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.438636:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b78a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.438778:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507211904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26373us (26588us total) trans 4294971870 rc 0/0 00000100:00100000:3.0:1789760730.438788:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4580 00000100:00100000:3.0:1789760730.438797:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711682782943 time 711727690064 state 0x2 ref 0 00000001:00080000:3.0:1789760730.438835:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971870 is committed 00000100:00100000:0.0:1789760730.445850:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.446198:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212032 00000100:00100000:2.0:1789760730.446209:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212032 00000100:00100000:2.0:1789760730.446265:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711682782943 time 711727690064 state 0x0 ref 1 00000100:00100000:2.0:1789760730.446280:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711727950480 time 711727690064 state 0x1 ref 1 00000100:00100000:2.0:1789760730.446310:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711727950480 time 711761613472 state 0x0 ref 1 00000100:00100000:2.0:1789760730.446333:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507212032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.446385:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18640896, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.446395:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.446415:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.446420:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.446426:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.446441:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.484399:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18640896 -> 18644992 00010000:00010000:2.0:1789760730.484424:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484437:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484452:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484460:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.484466:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484480:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484498:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.484512:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.484629:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507212032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38293us (38784us total) trans 4294971871 rc 0/0 00000100:00100000:2.0:1789760730.484643:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4581 00000100:00100000:2.0:1789760730.484653:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711727950480 time 711761613472 state 0x2 ref 0 00000001:00080000:3.0:1789760730.487545:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971871 is committed 00000100:00100000:0.0:1789760730.489007:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.489159:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212160 00000100:00100000:2.0:1789760730.489167:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212160 00000100:00100000:2.0:1789760730.489222:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711727950480 time 711761613472 state 0x0 ref 1 00000100:00100000:2.0:1789760730.489237:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711761873888 time 711761613472 state 0x1 ref 1 00000100:00100000:2.0:1789760730.489267:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711761873888 time 711804570260 state 0x0 ref 1 00000100:00100000:2.0:1789760730.489278:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507212160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.489325:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18644992, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.489333:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.489349:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.489353:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.489359:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.489372:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.529727:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971872 is committed 00002000:00010000:3.0:1789760730.530012:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18644992 -> 18649088 00010000:00010000:3.0:1789760730.530029:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530042:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530052:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530108:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.530113:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530122:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530143:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.530151:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.530277:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507212160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40995us (41275us total) trans 4294971872 rc 0/0 00000100:00100000:3.0:1789760730.530287:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4582 00000100:00100000:3.0:1789760730.530295:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711761873888 time 711804570260 state 0x2 ref 0 00000100:00100000:0.0:1789760730.533806:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760730.533899:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212288 00000100:00100000:3.0:1789760730.533909:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212288 00000100:00100000:3.0:1789760730.534014:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711761873888 time 711804570260 state 0x0 ref 1 00000100:00100000:3.0:1789760730.534028:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711804830676 time 711804570260 state 0x1 ref 1 00000100:00100000:3.0:1789760730.534145:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711804830676 time 711849446487 state 0x0 ref 1 00000100:00100000:3.0:1789760730.534158:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507212288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760730.534206:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18649088, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:3.0:1789760730.534214:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.534228:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760730.534232:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.534238:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.534253:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.607113:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971873 is committed 00002000:00010000:2.0:1789760730.609200:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18649088 -> 18653184 00010000:00010000:2.0:1789760730.609220:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609233:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609242:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609248:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.609253:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609261:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609284:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.609291:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b78ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.609394:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507212288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75233us (75591us total) trans 4294971873 rc 0/0 00000100:00100000:2.0:1789760730.609404:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4583 00000100:00100000:2.0:1789760730.609414:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711804830676 time 711849446487 state 0x2 ref 0 00000100:00100000:0.0:1789760730.612634:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.612741:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212416 00000100:00100000:2.0:1789760730.612751:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212416 00000100:00100000:2.0:1789760730.612805:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711804830676 time 711849446487 state 0x0 ref 1 00000100:00100000:2.0:1789760730.612820:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711849706903 time 711849446487 state 0x1 ref 1 00000100:00100000:2.0:1789760730.612850:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711849706903 time 711928153404 state 0x0 ref 1 00000100:00100000:2.0:1789760730.612863:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507212416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.612929:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18653184, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.612938:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.612953:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760730.612957:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.612963:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.612975:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.643996:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971874 is committed 00002000:00010000:2.0:1789760730.644455:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18653184 -> 18657280 00010000:00010000:2.0:1789760730.644470:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644482:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644491:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644497:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.644502:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644511:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644528:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.644535:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.644625:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507212416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31760us (31997us total) trans 4294971874 rc 0/0 00000100:00100000:2.0:1789760730.644634:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4584 00000100:00100000:2.0:1789760730.644642:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711849706903 time 711928153404 state 0x2 ref 0 00000100:00100000:0.0:1789760730.657807:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760730.657920:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212544 00000100:00100000:3.0:1789760730.657930:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212544 00000100:00100000:3.0:1789760730.657979:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711849706903 time 711928153404 state 0x0 ref 1 00000100:00100000:3.0:1789760730.657992:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711928413820 time 711928153404 state 0x1 ref 1 00000100:00100000:3.0:1789760730.658011:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711928413820 time 711973315135 state 0x0 ref 1 00000100:00100000:3.0:1789760730.658021:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760730.658078:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18657280, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:3.0:1789760730.658087:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.658102:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760730.658106:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.658112:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.658124:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.706395:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971875 is committed 00002000:00010000:3.0:1789760730.706484:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18657280 -> 18661376 00010000:00010000:3.0:1789760730.706498:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706510:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706519:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706525:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.706528:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706537:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706563:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.706570:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.706730:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48704us (48927us total) trans 4294971875 rc 0/0 00000100:00100000:3.0:1789760730.706738:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4585 00000100:00100000:3.0:1789760730.706747:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711928413820 time 711973315135 state 0x2 ref 0 00000100:00100000:1.0:1789760730.711326:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.711396:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212672 00000100:00100000:2.0:1789760730.711405:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212672 00000100:00100000:2.0:1789760730.711458:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711928413820 time 711973315135 state 0x0 ref 1 00000100:00100000:2.0:1789760730.711475:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711973575551 time 711973315135 state 0x1 ref 1 00000100:00100000:2.0:1789760730.711503:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711973575551 time 712026806520 state 0x0 ref 1 00000100:00100000:2.0:1789760730.711513:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.711558:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18661376, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.711566:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.711582:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760730.711586:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.711592:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.711606:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.747780:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971876 is committed 00002000:00010000:1.0:1789760730.747898:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18661376 -> 18665472 00010000:00010000:1.0:1789760730.747918:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747931:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747941:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747948:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760730.747953:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747962:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747980:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.747987:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b78cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.748069:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36552us (36749us total) trans 4294971876 rc 0/0 00000100:00100000:1.0:1789760730.748079:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4586 00000100:00100000:1.0:1789760730.748088:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711973575551 time 712026806520 state 0x2 ref 0 00000100:00100000:1.0:1789760730.755424:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.755520:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212800 00000100:00100000:2.0:1789760730.755530:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212800 00000100:00100000:2.0:1789760730.755587:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 711973575551 time 712026806520 state 0x0 ref 1 00000100:00100000:2.0:1789760730.755603:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712027066936 time 712026806520 state 0x1 ref 1 00000100:00100000:2.0:1789760730.755667:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712027066936 time 712070968821 state 0x0 ref 1 00000100:00100000:2.0:1789760730.755681:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.755751:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18665472, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.755760:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.755776:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760730.755781:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.755787:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.755803:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.781106:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971877 is committed 00002000:00010000:3.0:1789760730.781269:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18665472 -> 18669568 00010000:00010000:3.0:1789760730.781285:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781299:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781308:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781315:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.781319:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781334:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781353:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.781360:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b78d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.781454:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25772us (26035us total) trans 4294971877 rc 0/0 00000100:00100000:3.0:1789760730.781464:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4587 00000100:00100000:3.0:1789760730.781474:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712027066936 time 712070968821 state 0x2 ref 0 00000100:00100000:3.0:1789760730.796994:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.797481:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507212928 00000100:00100000:2.0:1789760730.797492:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507212928 00000100:00100000:2.0:1789760730.797549:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712027066936 time 712070968821 state 0x0 ref 1 00000100:00100000:2.0:1789760730.797566:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712071229237 time 712070968821 state 0x1 ref 1 00000100:00100000:2.0:1789760730.797600:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712071229237 time 712112902793 state 0x0 ref 1 00000100:00100000:2.0:1789760730.797612:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.797660:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18669568, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.797670:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.797685:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760730.797690:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.797722:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.797736:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.928540:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971878 is committed 00002000:00010000:2.0:1789760730.930158:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18669568 -> 18673664 00010000:00010000:2.0:1789760730.930191:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930205:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930214:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930221:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760730.930225:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930234:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930254:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.930261:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.930365:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507212928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 132750us (133377us total) trans 4294971878 rc 0/0 00000100:00100000:2.0:1789760730.930376:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4588 00000100:00100000:2.0:1789760730.930385:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712071229237 time 712112902793 state 0x2 ref 0 00000100:00100000:1.0:1789760730.935268:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760730.935341:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213056 00000100:00100000:2.0:1789760730.935351:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213056 00000100:00100000:2.0:1789760730.935406:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712071229237 time 712112902793 state 0x0 ref 1 00000100:00100000:2.0:1789760730.935421:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712113163209 time 712112902793 state 0x1 ref 1 00000100:00100000:2.0:1789760730.935456:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712113163209 time 712250759702 state 0x0 ref 1 00000100:00100000:2.0:1789760730.935468:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760730.935512:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18673664, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760730.935520:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.935534:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760730.935538:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.935544:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.935557:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.980125:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971879 is committed 00002000:00010000:3.0:1789760730.980289:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18673664 -> 18677760 00010000:00010000:3.0:1789760730.980305:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980377:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980387:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980399:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760730.980404:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980413:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980431:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.980438:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b78e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.980526:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45055us (45262us total) trans 4294971879 rc 0/0 00000100:00100000:3.0:1789760730.980535:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4589 00000100:00100000:3.0:1789760730.980543:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712113163209 time 712250759702 state 0x2 ref 0 00000100:00100000:0.0:1789760730.987102:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760730.987340:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213184 00000100:00100000:3.0:1789760730.987355:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213184 00000100:00100000:3.0:1789760730.987440:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712113163209 time 712250759702 state 0x0 ref 1 00000100:00100000:3.0:1789760730.987461:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712251020118 time 712250759702 state 0x1 ref 1 00000100:00100000:3.0:1789760730.987496:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712251020118 time 712302797895 state 0x0 ref 1 00000100:00100000:3.0:1789760730.987530:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760730.987607:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18677760, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:3.0:1789760730.987617:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.987639:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760730.987645:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.987653:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.987671:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.035850:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760731.046630:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18677760 -> 18681856 00010000:00010000:2.0:1789760731.046656:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046671:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046681:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046690:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760731.046729:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046741:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046765:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.046773:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b78eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.047147:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971880 is committed 00000100:00100000:2.0:1789760731.047215:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59681us (60119us total) trans 4294971880 rc 0/0 00000100:00100000:2.0:1789760731.047232:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4590 00000100:00100000:2.0:1789760731.047243:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712251020118 time 712302797895 state 0x2 ref 0 00000100:00100000:0.0:1789760731.052986:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760731.053447:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213312 00000100:00100000:2.0:1789760731.053461:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213312 00000100:00100000:2.0:1789760731.053544:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712251020118 time 712302797895 state 0x0 ref 1 00000100:00100000:2.0:1789760731.053562:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712303058311 time 712302797895 state 0x1 ref 1 00000100:00100000:2.0:1789760731.053634:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712303058311 time 712368936000 state 0x0 ref 1 00000100:00100000:2.0:1789760731.053650:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760731.053749:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18681856, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9007 00010000:00010000:2.0:1789760731.053760:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.053781:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760731.053786:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.053794:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.053810:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760731.054423:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760731.091742:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971881 is committed 00002000:00010000:2.0:1789760731.092116:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18681856 -> 18685952 00002000:00010000:2.0:1789760731.092122:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:2.0:1789760731.092126:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00010000:00010000:2.0:1789760731.092143:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092158:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092167:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092174:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760731.092180:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092190:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092215:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.092223:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b78f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.092349:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38694us (39369us total) trans 4294971881 rc 0/0 00000100:00100000:2.0:1789760731.092361:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4591 00000100:00100000:2.0:1789760731.092374:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712303058311 time 712368936000 state 0x2 ref 0 00000100:00100000:0.0:1789760731.099097:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.099439:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213440 00000100:00100000:3.0:1789760731.099453:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213440 00000100:00100000:3.0:1789760731.099533:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712303058311 time 712368936000 state 0x0 ref 1 00000100:00100000:3.0:1789760731.099552:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712369196416 time 712368936000 state 0x1 ref 1 00000100:00100000:3.0:1789760731.099644:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712369196416 time 712414944222 state 0x0 ref 1 00000100:00100000:3.0:1789760731.099658:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.099779:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18685952, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.099790:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.099811:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760731.099816:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.099822:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.099839:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.150102:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971882 is committed 00002000:00010000:3.0:1789760731.150224:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18685952 -> 18690048 00010000:00010000:3.0:1789760731.150243:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150258:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150268:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150276:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.150281:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150292:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150318:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.150329:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b78f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.150456:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50792us (51366us total) trans 4294971882 rc 0/0 00000100:00100000:3.0:1789760731.150468:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4592 00000100:00100000:3.0:1789760731.150481:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712369196416 time 712414944222 state 0x2 ref 0 00000100:00100000:0.0:1789760731.153518:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760731.154024:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213568 00000100:00100000:2.0:1789760731.154043:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213568 00000100:00100000:2.0:1789760731.154180:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712369196416 time 712414944222 state 0x0 ref 1 00000100:00100000:2.0:1789760731.154277:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712415204638 time 712414944222 state 0x1 ref 1 00000100:00100000:2.0:1789760731.154315:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712415204638 time 712469617664 state 0x0 ref 1 00000100:00100000:2.0:1789760731.154330:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760731.154397:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18690048, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:2.0:1789760731.154408:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.154497:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760731.154503:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.154510:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.154527:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.201542:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971883 is committed 00002000:00010000:3.0:1789760731.201846:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18690048 -> 18694144 00010000:00010000:3.0:1789760731.201973:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.201990:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.202000:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.202008:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.202014:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.202026:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.202123:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.202133:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7900 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.202261:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47928us (48752us total) trans 4294971883 rc 0/0 00000100:00100000:3.0:1789760731.202273:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4593 00000100:00100000:3.0:1789760731.202285:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712415204638 time 712469617664 state 0x2 ref 0 00000100:00100000:1.0:1789760731.207260:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760731.207398:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213696 00000100:00100000:2.0:1789760731.207412:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213696 00000100:00100000:2.0:1789760731.207505:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712415204638 time 712469617664 state 0x0 ref 1 00000100:00100000:2.0:1789760731.207523:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712469878080 time 712469617664 state 0x1 ref 1 00000100:00100000:2.0:1789760731.207564:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712469878080 time 712522866322 state 0x0 ref 1 00000100:00100000:2.0:1789760731.207579:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507213696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760731.207644:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18694144, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:2.0:1789760731.207655:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.207675:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760731.207681:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.207688:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.207739:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.254630:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971884 is committed 00002000:00010000:2.0:1789760731.254773:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18694144 -> 18698240 00010000:00010000:2.0:1789760731.254793:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254808:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254818:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254827:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760731.254833:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254842:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254876:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.254884:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7907 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.255070:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507213696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47486us (47816us total) trans 4294971884 rc 0/0 00000100:00100000:2.0:1789760731.255082:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4594 00000100:00100000:2.0:1789760731.255094:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712469878080 time 712522866322 state 0x2 ref 0 00000100:00100000:1.0:1789760731.259173:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.259303:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213824 00000100:00100000:3.0:1789760731.259326:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213824 00000100:00100000:3.0:1789760731.259403:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712469878080 time 712522866322 state 0x0 ref 1 00000100:00100000:3.0:1789760731.259422:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712523126738 time 712522866322 state 0x1 ref 1 00000100:00100000:3.0:1789760731.259457:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712523126738 time 712574758966 state 0x0 ref 1 00000100:00100000:3.0:1789760731.259471:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.259550:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18698240, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.259562:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.259583:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760731.259590:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.259598:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.259615:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.305245:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971885 is committed 00002000:00010000:3.0:1789760731.305399:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18698240 -> 18702336 00010000:00010000:3.0:1789760731.305420:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305436:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305447:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305454:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.305460:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305470:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305496:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.305512:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b790e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.305639:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46163us (46473us total) trans 4294971885 rc 0/0 00000100:00100000:3.0:1789760731.305651:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4595 00000100:00100000:3.0:1789760731.305662:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712523126738 time 712574758966 state 0x2 ref 0 00000100:00100000:3.0:1789760731.308838:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760731.309051:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507213952 00000100:00100000:2.0:1789760731.309064:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507213952 00000100:00100000:2.0:1789760731.309139:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712523126738 time 712574758966 state 0x0 ref 1 00000100:00100000:2.0:1789760731.309159:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712575019382 time 712574758966 state 0x1 ref 1 00000100:00100000:2.0:1789760731.309197:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712575019382 time 712624499156 state 0x0 ref 1 00000100:00100000:2.0:1789760731.309212:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760731.309281:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18702336, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:2.0:1789760731.309292:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.309323:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760731.309330:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.309338:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.309355:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.342243:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971886 is committed 00002000:00010000:2.0:1789760731.342336:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18702336 -> 18706432 00010000:00010000:2.0:1789760731.342355:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342371:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342382:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342390:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760731.342395:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342411:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342438:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.342447:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7915 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.342573:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507213952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33356us (33743us total) trans 4294971886 rc 0/0 00000100:00100000:2.0:1789760731.342585:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4596 00000100:00100000:2.0:1789760731.342598:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712575019382 time 712624499156 state 0x2 ref 0 00000100:00100000:2.0:1789760731.346463:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760731.346813:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214080 00000100:00100000:0.0:1789760731.346826:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214080 00000100:00100000:0.0:1789760731.347037:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712575019382 time 712624499156 state 0x0 ref 1 00000100:00100000:0.0:1789760731.347059:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712624759572 time 712624499156 state 0x1 ref 1 00000100:00100000:1.0:1789760731.347257:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712624759572 time 712662555857 state 0x0 ref 1 00000100:00100000:1.0:1789760731.347276:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507214080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760731.347354:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18706432, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:1.0:1789760731.347366:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.347387:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760731.347393:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.347400:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.347417:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.388072:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18706432 -> 18710528 00010000:00010000:1.0:1789760731.388099:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388115:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388127:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388143:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760731.388149:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388158:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388295:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.388304:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b791c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.388564:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507214080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41285us (42108us total) trans 4294971887 rc 0/0 00000100:00100000:1.0:1789760731.388577:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4597 00000100:00100000:1.0:1789760731.388589:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712624759572 time 712662555857 state 0x2 ref 0 00000001:00080000:0.0:1789760731.388689:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971887 is committed 00000100:00100000:1.0:1789760731.392251:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.392405:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214208 00000100:00100000:3.0:1789760731.392420:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214208 00000100:00100000:3.0:1789760731.392499:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712624759572 time 712662555857 state 0x0 ref 1 00000100:00100000:3.0:1789760731.392518:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712662816273 time 712662555857 state 0x1 ref 1 00000100:00100000:3.0:1789760731.392583:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712662816273 time 712707885486 state 0x0 ref 1 00000100:00100000:3.0:1789760731.392597:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507214208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.392663:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18710528, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.392675:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.392726:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760731.392732:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.392740:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.392757:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.428358:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18710528 -> 18714624 00010000:00010000:1.0:1789760731.428385:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428400:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428415:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428424:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760731.428430:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428440:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428464:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.428472:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7923 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.428600:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507214208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35997us (36357us total) trans 4294971888 rc 0/0 00000100:00100000:1.0:1789760731.428612:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4598 00000100:00100000:1.0:1789760731.428625:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712662816273 time 712707885486 state 0x2 ref 0 00000001:00080000:2.0:1789760731.429742:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971888 is committed 00000100:00100000:2.0:1789760731.433312:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.433490:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214336 00000100:00100000:3.0:1789760731.433504:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214336 00000100:00100000:3.0:1789760731.433583:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712662816273 time 712707885486 state 0x0 ref 1 00000100:00100000:3.0:1789760731.433604:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712708145902 time 712707885486 state 0x1 ref 1 00000100:00100000:3.0:1789760731.434009:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712708145902 time 712749308869 state 0x0 ref 1 00000100:00100000:3.0:1789760731.434033:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507214336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.434102:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18714624, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.434113:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.434135:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760731.434141:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.434148:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.434167:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.470020:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971889 is committed 00002000:00010000:0.0:1789760731.471958:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18714624 -> 18718720 00010000:00010000:0.0:1789760731.473083:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473105:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473116:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473125:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760731.473130:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473141:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473170:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.473180:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b792a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.473317:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507214336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39285us (40013us total) trans 4294971889 rc 0/0 00000100:00100000:0.0:1789760731.473329:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4599 00000100:00100000:0.0:1789760731.473341:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712708145902 time 712749308869 state 0x2 ref 0 00000100:00100000:0.0:1789760731.476797:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760731.477047:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214464 00000100:00100000:1.0:1789760731.477061:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214464 00000100:00100000:1.0:1789760731.477142:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712708145902 time 712749308869 state 0x0 ref 1 00000100:00100000:1.0:1789760731.477163:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712749569285 time 712749308869 state 0x1 ref 1 00000100:00100000:1.0:1789760731.477253:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712749569285 time 712792555116 state 0x0 ref 1 00000100:00100000:1.0:1789760731.477269:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760731.477339:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18718720, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:1.0:1789760731.477351:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.477372:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760731.477378:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.477387:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.477407:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.523668:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971890 is committed 00002000:00010000:0.0:1789760731.523809:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18718720 -> 18722816 00010000:00010000:0.0:1789760731.523827:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523839:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523846:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523851:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760731.523855:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523873:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523906:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.523915:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7931 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.524034:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46761us (47243us total) trans 4294971890 rc 0/0 00000100:00100000:0.0:1789760731.524046:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4600 00000100:00100000:0.0:1789760731.524059:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712749569285 time 712792555116 state 0x2 ref 0 00000100:00100000:1.0:1789760731.528431:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.528565:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214592 00000100:00100000:3.0:1789760731.528578:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214592 00000100:00100000:3.0:1789760731.528647:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712749569285 time 712792555116 state 0x0 ref 1 00000100:00100000:3.0:1789760731.529275:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712792815532 time 712792555116 state 0x1 ref 1 00000100:00100000:1.0:1789760731.529384:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712792815532 time 712844683674 state 0x0 ref 1 00000100:00100000:1.0:1789760731.529403:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507214592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760731.529474:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18722816, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:1.0:1789760731.529485:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.529505:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760731.529510:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.529519:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.529536:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.563636:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18722816 -> 18726912 00010000:00010000:3.0:1789760731.563666:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563682:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563729:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563739:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.563744:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563754:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563780:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.563789:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7938 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.563922:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507214592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34518us (35501us total) trans 4294971891 rc 0/0 00000100:00100000:3.0:1789760731.563935:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4601 00000100:00100000:3.0:1789760731.563947:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712792815532 time 712844683674 state 0x2 ref 0 00000001:00080000:0.0:1789760731.564102:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971891 is committed 00000100:00100000:2.0:1789760731.573019:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760731.573177:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214720 00000100:00100000:0.0:1789760731.573192:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214720 00000100:00100000:0.0:1789760731.573269:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712792815532 time 712844683674 state 0x0 ref 1 00000100:00100000:0.0:1789760731.573287:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712844944090 time 712844683674 state 0x1 ref 1 00000100:00100000:0.0:1789760731.573324:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712844944090 time 712888626579 state 0x0 ref 1 00000100:00100000:0.0:1789760731.573339:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760731.573414:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18726912, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:0.0:1789760731.573426:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.573448:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760731.573454:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.573462:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.573480:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.615138:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971892 is committed 00002000:00010000:2.0:1789760731.615278:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18726912 -> 18731008 00010000:00010000:2.0:1789760731.615300:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615316:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615325:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615333:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760731.615339:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615349:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615377:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.615387:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b793f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.615566:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42222us (42555us total) trans 4294971892 rc 0/0 00000100:00100000:2.0:1789760731.615579:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4602 00000100:00100000:2.0:1789760731.615602:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712844944090 time 712888626579 state 0x2 ref 0 00000100:00100000:1.0:1789760731.620987:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760731.621176:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214848 00000100:00100000:0.0:1789760731.621190:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214848 00000100:00100000:0.0:1789760731.621272:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712844944090 time 712888626579 state 0x0 ref 1 00000100:00100000:0.0:1789760731.621297:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712888886995 time 712888626579 state 0x1 ref 1 00000100:00100000:0.0:1789760731.621329:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712888886995 time 712936632782 state 0x0 ref 1 00000100:00100000:0.0:1789760731.621344:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760731.621420:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18731008, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:0.0:1789760731.621431:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.621457:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760731.621464:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.621471:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.621490:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.661996:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971893 is committed 00002000:00010000:1.0:1789760731.662129:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18731008 -> 18735104 00010000:00010000:1.0:1789760731.662151:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662167:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662177:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662185:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760731.662191:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662202:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662228:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.662253:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7946 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.662384:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41027us (41397us total) trans 4294971893 rc 0/0 00000100:00100000:1.0:1789760731.662399:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4603 00000100:00100000:1.0:1789760731.662412:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712888886995 time 712936632782 state 0x2 ref 0 00000100:00100000:3.0:1789760731.668639:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760731.668750:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507214976 00000100:00100000:1.0:1789760731.668776:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507214976 00000100:00100000:1.0:1789760731.668852:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712888886995 time 712936632782 state 0x0 ref 1 00000100:00100000:1.0:1789760731.668883:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712936893198 time 712936632782 state 0x1 ref 1 00000100:00100000:1.0:1789760731.668915:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712936893198 time 712984218221 state 0x0 ref 1 00000100:00100000:1.0:1789760731.668931:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760731.669003:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18735104, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:1.0:1789760731.669015:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.669036:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760731.669041:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.669049:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.669073:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.713573:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971894 is committed 00002000:00010000:1.0:1789760731.713678:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18735104 -> 18739200 00010000:00010000:1.0:1789760731.713723:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713738:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713746:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713751:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760731.713755:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713762:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713787:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713792:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b794d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.713928:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507214976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44993us (45298us total) trans 4294971894 rc 0/0 00000100:00100000:1.0:1789760731.713941:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4604 00000100:00100000:1.0:1789760731.713953:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712936893198 time 712984218221 state 0x2 ref 0 00000100:00100000:0.0:1789760731.721142:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.721360:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215104 00000100:00100000:3.0:1789760731.721546:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215104 00000100:00100000:3.0:1789760731.721636:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712936893198 time 712984218221 state 0x0 ref 1 00000100:00100000:3.0:1789760731.721657:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712984478637 time 712984218221 state 0x1 ref 1 00000100:00100000:3.0:1789760731.721719:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712984478637 time 713037022214 state 0x0 ref 1 00000100:00100000:3.0:1789760731.721734:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507215104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.721799:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18739200, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.721810:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.721829:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760731.721835:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.721843:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.721875:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.793577:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18739200 -> 18743296 00010000:00010000:3.0:1789760731.793598:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793613:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793623:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793631:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.793635:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793643:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793663:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.793671:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7954 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.794160:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507215104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72424us (73030us total) trans 4294971895 rc 0/0 00000100:00100000:3.0:1789760731.794172:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4605 00000100:00100000:3.0:1789760731.794183:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712984478637 time 713037022214 state 0x2 ref 0 00000001:00080000:3.0:1789760731.794228:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971895 is committed 00000100:00100000:1.0:1789760731.799226:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760731.799421:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215232 00000100:00100000:0.0:1789760731.799437:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215232 00000100:00100000:0.0:1789760731.799505:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 712984478637 time 713037022214 state 0x0 ref 1 00000100:00100000:0.0:1789760731.799521:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713037282630 time 713037022214 state 0x1 ref 1 00000100:00100000:0.0:1789760731.799554:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713037282630 time 713114856896 state 0x0 ref 1 00000100:00100000:0.0:1789760731.799569:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760731.799618:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18743296, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:0.0:1789760731.799628:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.799645:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760731.799650:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.799657:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.799671:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.843881:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18743296 -> 18747392 00010000:00010000:3.0:1789760731.843912:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843925:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843935:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843943:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.843948:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843957:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843976:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.843989:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b795b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.844080:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507215232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44509us (44863us total) trans 4294971896 rc 0/0 00000100:00100000:3.0:1789760731.844090:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4606 00000100:00100000:3.0:1789760731.844100:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713037282630 time 713114856896 state 0x2 ref 0 00000001:00080000:3.0:1789760731.844136:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971896 is committed 00000100:00100000:1.0:1789760731.848189:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.848270:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215360 00000100:00100000:3.0:1789760731.848280:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215360 00000100:00100000:3.0:1789760731.848331:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713037282630 time 713114856896 state 0x0 ref 1 00000100:00100000:3.0:1789760731.848347:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713115117312 time 713114856896 state 0x1 ref 1 00000100:00100000:3.0:1789760731.848409:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713115117312 time 713163712207 state 0x0 ref 1 00000100:00100000:3.0:1789760731.848421:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507215360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.848476:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18747392, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.848487:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.848502:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760731.848506:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.848513:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.848528:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.888520:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18747392 -> 18751488 00010000:00010000:3.0:1789760731.888539:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888553:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888562:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888569:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760731.888574:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888583:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888602:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.888609:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7962 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.888739:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507215360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40314us (40554us total) trans 4294971897 rc 0/0 00000100:00100000:3.0:1789760731.888749:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4607 00000100:00100000:3.0:1789760731.888758:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713115117312 time 713163712207 state 0x2 ref 0 00000001:00080000:3.0:1789760731.888796:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971897 is committed 00000100:00100000:0.0:1789760731.896068:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.896177:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215488 00000100:00100000:3.0:1789760731.896188:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215488 00000100:00100000:3.0:1789760731.896252:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713115117312 time 713163712207 state 0x0 ref 1 00000100:00100000:3.0:1789760731.896439:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713163972623 time 713163712207 state 0x1 ref 1 00000100:00100000:3.0:1789760731.896474:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713163972623 time 713211776688 state 0x0 ref 1 00000100:00100000:3.0:1789760731.896486:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760731.896874:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18751488, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:2.0:1789760731.896886:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.896912:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760731.896917:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.896923:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.896938:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.953183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971898 is committed 00002000:00010000:0.0:1789760731.953794:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18751488 -> 18755584 00010000:00010000:0.0:1789760731.953810:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953822:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953832:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953839:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760731.953846:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953856:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953885:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953903:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7969 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.953997:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57508us (57934us total) trans 4294971898 rc 0/0 00000100:00100000:0.0:1789760731.954008:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4608 00000100:00100000:0.0:1789760731.954018:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713163972623 time 713211776688 state 0x2 ref 0 00000100:00100000:0.0:1789760731.960739:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760731.962947:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215616 00000100:00100000:3.0:1789760731.962961:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215616 00000100:00100000:3.0:1789760731.963024:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713163972623 time 713211776688 state 0x0 ref 1 00000100:00100000:3.0:1789760731.963040:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713212037104 time 713211776688 state 0x1 ref 1 00000100:00100000:3.0:1789760731.963156:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713212037104 time 713278457421 state 0x0 ref 1 00000100:00100000:3.0:1789760731.963171:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760731.963218:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18755584, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760731.963226:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.963241:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760731.963246:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.963253:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.963268:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.015608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971899 is committed 00002000:00010000:1.0:1789760732.015867:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18755584 -> 18759680 00010000:00010000:1.0:1789760732.015884:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015896:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015904:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015910:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760732.015915:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015923:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015947:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.015954:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7970 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.016129:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52955us (55394us total) trans 4294971899 rc 0/0 00000100:00100000:1.0:1789760732.016139:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4609 00000100:00100000:1.0:1789760732.016148:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713212037104 time 713278457421 state 0x2 ref 0 00000100:00100000:1.0:1789760732.021258:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760732.021417:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215744 00000100:00100000:3.0:1789760732.021427:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215744 00000100:00100000:3.0:1789760732.021480:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713212037104 time 713278457421 state 0x0 ref 1 00000100:00100000:3.0:1789760732.021496:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713278717837 time 713278457421 state 0x1 ref 1 00000100:00100000:3.0:1789760732.021553:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713278717837 time 713336857174 state 0x0 ref 1 00000100:00100000:3.0:1789760732.021565:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507215744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760732.021611:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18759680, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0x9007 00010000:00010000:3.0:1789760732.021620:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.021634:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760732.021638:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.021644:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.021657:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.048247:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971900 is committed 00002000:00010000:1.0:1789760732.048328:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18759680 -> 18763776 00002000:00010000:1.0:1789760732.048333:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:1.0:1789760732.048335:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:1.0:1789760732.048349:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048362:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048371:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048378:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760732.048383:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048393:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048413:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.048420:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7977 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.048510:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507215744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26941us (27256us total) trans 4294971900 rc 0/0 00000100:00100000:1.0:1789760732.048519:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4610 00000100:00100000:1.0:1789760732.048529:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713278717837 time 713336857174 state 0x2 ref 0 00000100:00100000:0.0:1789760732.051660:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.051843:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507215872 00000100:00100000:2.0:1789760732.051853:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507215872 00000100:00100000:2.0:1789760732.051916:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713278717837 time 713336857174 state 0x0 ref 1 00000100:00100000:2.0:1789760732.051930:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713337117590 time 713336857174 state 0x1 ref 1 00000100:00100000:2.0:1789760732.051993:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713337117590 time 713367294923 state 0x0 ref 1 00000100:00100000:2.0:1789760732.052005:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760732.052058:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18763776, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:2.0:1789760732.052066:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052081:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760732.052085:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.052091:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052105:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.062662:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760732.091941:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971901 is committed 00002000:00010000:2.0:1789760732.092138:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18763776 -> 18767872 00010000:00010000:2.0:1789760732.092154:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092167:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092176:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092183:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.092188:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092197:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092216:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.092223:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b797e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.092316:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507215872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40307us (40660us total) trans 4294971901 rc 0/0 00000100:00100000:2.0:1789760732.092325:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4611 00000100:00100000:2.0:1789760732.092335:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713337117590 time 713367294923 state 0x2 ref 0 00000100:00100000:0.0:1789760732.097119:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.097288:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216000 00000100:00100000:1.0:1789760732.097299:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216000 00000100:00100000:1.0:1789760732.097472:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713337117590 time 713367294923 state 0x0 ref 1 00000100:00100000:1.0:1789760732.097491:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713367555339 time 713367294923 state 0x1 ref 1 00000100:00100000:1.0:1789760732.097587:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713367555339 time 713412889892 state 0x0 ref 1 00000100:00100000:1.0:1789760732.097603:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.097652:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18767872, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.097661:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.097676:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760732.097681:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.097688:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.097745:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.160097:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971902 is committed 00002000:00010000:0.0:1789760732.161807:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18767872 -> 18771968 00010000:00010000:0.0:1789760732.161826:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161838:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161847:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161853:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760732.162028:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.162040:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.162060:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.162069:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7985 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.162478:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64873us (65364us total) trans 4294971902 rc 0/0 00000100:00100000:0.0:1789760732.162491:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4612 00000100:00100000:0.0:1789760732.162501:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713367555339 time 713412889892 state 0x2 ref 0 00000100:00100000:3.0:1789760732.174146:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.174248:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216128 00000100:00100000:1.0:1789760732.174258:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216128 00000100:00100000:1.0:1789760732.174318:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713367555339 time 713412889892 state 0x0 ref 1 00000100:00100000:1.0:1789760732.174334:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713413150308 time 713412889892 state 0x1 ref 1 00000100:00100000:1.0:1789760732.174370:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713413150308 time 713489672627 state 0x0 ref 1 00000100:00100000:1.0:1789760732.174383:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.174436:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18771968, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.174448:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.174464:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760732.174469:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.174476:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.174491:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.209220:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971903 is committed 00002000:00010000:1.0:1789760732.209311:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18771968 -> 18776064 00010000:00010000:1.0:1789760732.209327:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209341:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209351:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209358:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760732.209363:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209373:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209394:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.209402:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b798c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.209487:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35101us (35345us total) trans 4294971903 rc 0/0 00000100:00100000:1.0:1789760732.209498:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4613 00000100:00100000:1.0:1789760732.209507:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713413150308 time 713489672627 state 0x2 ref 0 00000100:00100000:2.0:1789760732.216533:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760732.216657:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216256 00000100:00100000:0.0:1789760732.216668:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216256 00000100:00100000:0.0:1789760732.216972:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713413150308 time 713489672627 state 0x0 ref 1 00000100:00100000:0.0:1789760732.216989:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713489933043 time 713489672627 state 0x1 ref 1 00000100:00100000:0.0:1789760732.217062:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713489933043 time 713532364594 state 0x0 ref 1 00000100:00100000:0.0:1789760732.217076:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760732.217126:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18776064, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:0.0:1789760732.217136:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.217153:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760732.217157:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.217164:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.217179:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.258596:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971904 is committed 00002000:00010000:2.0:1789760732.258815:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18776064 -> 18780160 00010000:00010000:2.0:1789760732.258830:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.258844:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.258853:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.258868:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.258873:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.258883:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.259054:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.259065:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7993 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.259157:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42080us (42630us total) trans 4294971904 rc 0/0 00000100:00100000:2.0:1789760732.259171:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4614 00000100:00100000:2.0:1789760732.259181:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713489933043 time 713532364594 state 0x2 ref 0 00000100:00100000:0.0:1789760732.263188:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.263344:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216384 00000100:00100000:1.0:1789760732.263355:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216384 00000100:00100000:1.0:1789760732.263411:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713489933043 time 713532364594 state 0x0 ref 1 00000100:00100000:1.0:1789760732.263427:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713532625010 time 713532364594 state 0x1 ref 1 00000100:00100000:1.0:1789760732.263457:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713532625010 time 713578760707 state 0x0 ref 1 00000100:00100000:1.0:1789760732.263471:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.263519:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18780160, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.263528:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.263545:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760732.263550:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.263557:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.263570:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.310720:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971905 is committed 00002000:00010000:3.0:1789760732.310852:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18780160 -> 18784256 00010000:00010000:3.0:1789760732.310877:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310889:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310907:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310914:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760732.310919:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310927:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310946:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.310954:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b799a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.311134:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47659us (47950us total) trans 4294971905 rc 0/0 00000100:00100000:3.0:1789760732.311145:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4615 00000100:00100000:3.0:1789760732.311155:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713532625010 time 713578760707 state 0x2 ref 0 00000100:00100000:3.0:1789760732.320372:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.320483:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216512 00000100:00100000:1.0:1789760732.320500:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216512 00000100:00100000:1.0:1789760732.320572:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713532625010 time 713578760707 state 0x0 ref 1 00000100:00100000:1.0:1789760732.320589:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713579021123 time 713578760707 state 0x1 ref 1 00000100:00100000:1.0:1789760732.320622:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713579021123 time 713635925313 state 0x0 ref 1 00000100:00100000:1.0:1789760732.320636:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.320688:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18784256, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.320749:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.320777:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760732.320784:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.320791:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.320808:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.359194:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18784256 -> 18788352 00010000:00010000:1.0:1789760732.359221:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.359242:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.359252:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.359261:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760732.359266:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.359277:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.359407:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971906 is committed 00010000:00010000:1.0:1789760732.359464:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.359475:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.359597:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38957us (39235us total) trans 4294971906 rc 0/0 00000100:00100000:1.0:1789760732.359609:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4616 00000100:00100000:1.0:1789760732.359621:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713579021123 time 713635925313 state 0x2 ref 0 00000100:00100000:0.0:1789760732.363458:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.363578:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216640 00000100:00100000:2.0:1789760732.363592:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216640 00000100:00100000:2.0:1789760732.363662:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713579021123 time 713635925313 state 0x0 ref 1 00000100:00100000:2.0:1789760732.363680:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713636185729 time 713635925313 state 0x1 ref 1 00000100:00100000:2.0:1789760732.363825:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713636185729 time 713679126561 state 0x0 ref 1 00000100:00100000:2.0:1789760732.363840:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760732.363922:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18788352, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:2.0:1789760732.363932:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.363951:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760732.363956:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.363963:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.363979:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.398741:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18788352 -> 18792448 00010000:00010000:0.0:1789760732.398769:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398784:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398795:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398804:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760732.398810:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398820:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398844:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.398853:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b79a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.398986:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507216640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35140us (35534us total) trans 4294971907 rc 0/0 00000100:00100000:0.0:1789760732.398999:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4617 00000100:00100000:0.0:1789760732.399011:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713636185729 time 713679126561 state 0x2 ref 0 00000001:00080000:1.0:1789760732.400957:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971907 is committed 00000100:00100000:0.0:1789760732.402521:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760732.402622:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216768 00000100:00100000:3.0:1789760732.402635:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216768 00000100:00100000:3.0:1789760732.402742:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713636185729 time 713679126561 state 0x0 ref 1 00000100:00100000:3.0:1789760732.402763:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713679386977 time 713679126561 state 0x1 ref 1 00000100:00100000:3.0:1789760732.402791:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713679386977 time 713718094198 state 0x0 ref 1 00000100:00100000:3.0:1789760732.402806:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760732.402884:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18792448, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:3.0:1789760732.403064:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.403086:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760732.403092:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.403099:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.403123:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.447497:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18792448 -> 18796544 00000001:00080000:3.0:1789760732.447514:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971908 is committed 00010000:00010000:2.0:1789760732.447528:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447546:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447557:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447565:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.447571:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447582:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447609:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.447618:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b79af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.447870:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45047us (45340us total) trans 4294971908 rc 0/0 00000100:00100000:3.0:1789760732.447885:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4618 00000100:00100000:3.0:1789760732.447897:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713679386977 time 713718094198 state 0x2 ref 0 00000100:00100000:3.0:1789760732.452534:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.452821:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507216896 00000100:00100000:1.0:1789760732.452834:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507216896 00000100:00100000:1.0:1789760732.452922:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713679386977 time 713718094198 state 0x0 ref 1 00000100:00100000:1.0:1789760732.452943:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713718354614 time 713718094198 state 0x1 ref 1 00000100:00100000:1.0:1789760732.452985:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713718354614 time 713768287480 state 0x0 ref 1 00000100:00100000:1.0:1789760732.453001:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.453071:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18796544, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.453083:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.453104:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760732.453110:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.453118:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.453136:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.523251:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971909 is committed 00002000:00010000:2.0:1789760732.524001:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18796544 -> 18800640 00010000:00010000:2.0:1789760732.524024:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524040:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524057:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524065:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.524071:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524081:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524108:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.524117:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b79b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.524312:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507216896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71298us (71777us total) trans 4294971909 rc 0/0 00000100:00100000:2.0:1789760732.524325:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4619 00000100:00100000:2.0:1789760732.524338:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713718354614 time 713768287480 state 0x2 ref 0 00000100:00100000:2.0:1789760732.536847:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760732.536935:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217024 00000100:00100000:0.0:1789760732.536949:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217024 00000100:00100000:0.0:1789760732.537031:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713718354614 time 713768287480 state 0x0 ref 1 00000100:00100000:0.0:1789760732.537053:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713768547896 time 713768287480 state 0x1 ref 1 00000100:00100000:0.0:1789760732.537093:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713768547896 time 713852396131 state 0x0 ref 1 00000100:00100000:0.0:1789760732.537109:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760732.537380:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18800640, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:0.0:1789760732.537391:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.537529:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760732.537537:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.537545:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.537563:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.594618:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971910 is committed 00002000:00010000:2.0:1789760732.594786:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18800640 -> 18804736 00010000:00010000:2.0:1789760732.594802:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594820:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594828:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594835:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.594839:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594847:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594905:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.594914:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b79bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.594996:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57886us (58161us total) trans 4294971910 rc 0/0 00000100:00100000:2.0:1789760732.595006:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4620 00000100:00100000:2.0:1789760732.595015:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713768547896 time 713852396131 state 0x2 ref 0 00000100:00100000:1.0:1789760732.599034:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760732.599495:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217152 00000100:00100000:3.0:1789760732.599514:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217152 00000100:00100000:3.0:1789760732.599595:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713768547896 time 713852396131 state 0x0 ref 1 00000100:00100000:3.0:1789760732.599615:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713852656547 time 713852396131 state 0x1 ref 1 00000100:00100000:3.0:1789760732.599653:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713852656547 time 713914956401 state 0x0 ref 1 00000100:00100000:3.0:1789760732.599668:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507217152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760732.599779:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18804736, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:3.0:1789760732.599791:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.600218:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760732.600226:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.600238:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.600258:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.645588:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971911 is committed 00002000:00010000:2.0:1789760732.645722:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18804736 -> 18808832 00010000:00010000:2.0:1789760732.645739:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645752:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645769:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645776:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.645781:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645791:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645809:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.645817:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b79c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.646534:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507217152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46861us (47502us total) trans 4294971911 rc 0/0 00000100:00100000:3.0:1789760732.646547:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4621 00000100:00100000:3.0:1789760732.646557:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713852656547 time 713914956401 state 0x2 ref 0 00000100:00100000:1.0:1789760732.654407:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.654582:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217280 00000100:00100000:2.0:1789760732.654592:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217280 00000100:00100000:2.0:1789760732.654648:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713852656547 time 713914956401 state 0x0 ref 1 00000100:00100000:2.0:1789760732.654666:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713915216817 time 713914956401 state 0x1 ref 1 00000100:00100000:2.0:1789760732.654735:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713915216817 time 713970031010 state 0x0 ref 1 00000100:00100000:2.0:1789760732.654747:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760732.654794:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18808832, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:2.0:1789760732.654803:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.654818:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760732.654823:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.654829:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.654843:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.691593:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971912 is committed 00002000:00010000:3.0:1789760732.691677:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18808832 -> 18812928 00010000:00010000:3.0:1789760732.691707:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691721:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691798:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691806:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760732.691811:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691825:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691866:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.691875:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b79cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.692002:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37252us (37599us total) trans 4294971912 rc 0/0 00000100:00100000:3.0:1789760732.692012:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4622 00000100:00100000:3.0:1789760732.692020:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713915216817 time 713970031010 state 0x2 ref 0 00000100:00100000:2.0:1789760732.697491:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760732.697570:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217408 00000100:00100000:0.0:1789760732.697647:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217408 00000100:00100000:0.0:1789760732.697723:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713915216817 time 713970031010 state 0x0 ref 1 00000100:00100000:0.0:1789760732.697813:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713970291426 time 713970031010 state 0x1 ref 1 00000100:00100000:0.0:1789760732.697838:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713970291426 time 714013141660 state 0x0 ref 1 00000100:00100000:0.0:1789760732.697851:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507217408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760732.697909:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18812928, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:0.0:1789760732.697919:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.697939:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760732.697944:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.697951:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.697965:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.764933:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971913 is committed 00002000:00010000:1.0:1789760732.765953:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18812928 -> 18817024 00010000:00010000:1.0:1789760732.765971:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.765989:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.765998:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.766004:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760732.766008:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.766017:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.766036:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.766044:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b79d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.766160:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507217408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68306us (68670us total) trans 4294971913 rc 0/0 00000100:00100000:1.0:1789760732.766170:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4623 00000100:00100000:1.0:1789760732.766179:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713970291426 time 714013141660 state 0x2 ref 0 00000100:00100000:0.0:1789760732.776307:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760732.776394:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217536 00000100:00100000:3.0:1789760732.776405:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217536 00000100:00100000:3.0:1789760732.776458:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 713970291426 time 714013141660 state 0x0 ref 1 00000100:00100000:3.0:1789760732.776475:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714013402076 time 714013141660 state 0x1 ref 1 00000100:00100000:3.0:1789760732.776509:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714013402076 time 714091812582 state 0x0 ref 1 00000100:00100000:3.0:1789760732.776521:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760732.776569:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18817024, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:3.0:1789760732.776577:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.776593:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760732.776597:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.776603:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.776617:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.815449:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971914 is committed 00002000:00010000:3.0:1789760732.815572:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18817024 -> 18821120 00010000:00010000:3.0:1789760732.815587:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815599:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815608:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815615:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760732.815620:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815628:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815654:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.815661:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.816222:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507217536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39697us (39918us total) trans 4294971914 rc 0/0 00000100:00100000:3.0:1789760732.816233:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4624 00000100:00100000:3.0:1789760732.816243:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714013402076 time 714091812582 state 0x2 ref 0 00000100:00100000:3.0:1789760732.825075:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.825295:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217664 00000100:00100000:1.0:1789760732.826008:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217664 00000100:00100000:1.0:1789760732.826077:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714013402076 time 714091812582 state 0x0 ref 1 00000100:00100000:1.0:1789760732.826092:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714092072998 time 714091812582 state 0x1 ref 1 00000100:00100000:1.0:1789760732.826132:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714092072998 time 714141434516 state 0x0 ref 1 00000100:00100000:1.0:1789760732.826145:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507217664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.826201:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18821120, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.826209:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.826225:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760732.826229:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.826236:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.826250:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.879671:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18821120 -> 18825216 00010000:00010000:2.0:1789760732.879715:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879730:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879739:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879747:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.879753:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879762:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879780:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.879789:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b79e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.879881:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507217664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53734us (55522us total) trans 4294971915 rc 0/0 00000100:00100000:2.0:1789760732.879901:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4625 00000100:00100000:2.0:1789760732.879910:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714092072998 time 714141434516 state 0x2 ref 0 00000001:00080000:2.0:1789760732.879948:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971915 is committed 00000100:00100000:2.0:1789760732.884820:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760732.886564:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217792 00000100:00100000:1.0:1789760732.886578:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217792 00000100:00100000:1.0:1789760732.886642:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714092072998 time 714141434516 state 0x0 ref 1 00000100:00100000:1.0:1789760732.886659:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714141694932 time 714141434516 state 0x1 ref 1 00000100:00100000:1.0:1789760732.886716:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714141694932 time 714202018241 state 0x0 ref 1 00000100:00100000:1.0:1789760732.886733:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507217792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760732.886784:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18825216, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760732.886792:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.886809:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760732.886814:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.886820:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.886834:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.889537:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.889573:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.889599:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.889619:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.889641:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.889669:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218048 00000100:00100000:2.0:1789760732.889678:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218048 00000100:00100000:2.0:1789760732.889754:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218304 00000100:00100000:2.0:1789760732.889758:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218304 00000100:00100000:2.0:1789760732.889774:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218176 00000100:00100000:2.0:1789760732.889776:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218176 00000100:00100000:2.0:1789760732.889787:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218432 00000100:00100000:2.0:1789760732.889789:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218432 00000100:00100000:2.0:1789760732.889805:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 868 00000100:00100000:2.0:1789760732.889814:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507218048:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.889888:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507218048:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 75us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.890265:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 868 00000100:00100000:2.0:1789760732.890288:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 869 00000100:00100000:2.0:1789760732.890296:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507218304:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.890325:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507218304:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 31us (752us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.890331:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 869 00000100:00100000:2.0:1789760732.890343:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 870 00000100:00100000:2.0:1789760732.890349:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507218176:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.891529:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507218176:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 1175us (1904us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.891539:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 870 00000100:00100000:2.0:1789760732.891563:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 871 00000100:00100000:2.0:1789760732.891572:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507218432:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.891598:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507218432:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 29us (1957us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.891605:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 871 00000100:00100000:2.0:1789760732.891629:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507217920 00000100:00100000:2.0:1789760732.891634:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507217920 00000100:00100000:2.0:1789760732.891668:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 464 00000100:00100000:2.0:1789760732.891674:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223496450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507217920:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.891730:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223496450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507217920:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 55us (2130us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.891736:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 464 00000100:00080000:2.0:1789760732.891768:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760732.891774:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1789760732.935514:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18825216 -> 18829312 00010000:00010000:2.0:1789760732.935534:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935548:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935557:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935565:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760732.935570:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935578:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935602:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.935609:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b79e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18825216->18829311] (req 18825216->18829311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.935727:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507217792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48963us (50882us total) trans 4294971916 rc 0/0 00000100:00100000:2.0:1789760732.935738:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4626 00000100:00100000:2.0:1789760732.935747:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714141694932 time 714202018241 state 0x2 ref 0 00000001:00080000:2.0:1789760732.935787:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971916 is committed 00000100:00100000:2.0:1789760732.940610:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760732.940756:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218560 00000100:00100000:3.0:1789760732.940774:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218560 00000100:00100000:3.0:1789760732.940870:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714141694932 time 714202018241 state 0x0 ref 1 00000100:00100000:3.0:1789760732.940901:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714202278657 time 714202018241 state 0x1 ref 1 00000100:00100000:3.0:1789760732.940936:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714202278657 time 714256239737 state 0x0 ref 1 00000100:00100000:3.0:1789760732.940952:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507218560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760732.941016:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18829312, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:3.0:1789760732.941029:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.941056:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760732.941061:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.941069:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.941087:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.982134:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18829312 -> 18833408 00010000:00010000:3.0:1789760732.982152:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.982164:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.982259:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.982267:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760732.982272:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.982282:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.982486:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971917 is committed 00010000:00010000:3.0:1789760732.982535:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.982546:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b79ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.982635:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507218560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41682us (42030us total) trans 4294971917 rc 0/0 00000100:00100000:3.0:1789760732.982645:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4627 00000100:00100000:3.0:1789760732.982655:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714202278657 time 714256239737 state 0x2 ref 0 00000100:00100000:0.0:1789760732.987541:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760732.987843:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218688 00000100:00100000:2.0:1789760732.987861:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218688 00000100:00100000:2.0:1789760732.987916:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714202278657 time 714256239737 state 0x0 ref 1 00000100:00100000:2.0:1789760732.987932:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714256500153 time 714256239737 state 0x1 ref 1 00000100:00100000:2.0:1789760732.987992:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714256500153 time 714303294351 state 0x0 ref 1 00000100:00100000:2.0:1789760732.988005:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507218688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760732.988054:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18833408, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:2.0:1789760732.988062:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.988076:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760732.988090:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.988097:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.988112:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.019161:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971918 is committed 00002000:00010000:1.0:1789760733.019446:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18833408 -> 18837504 00010000:00010000:1.0:1789760733.019463:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019477:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019487:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019494:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760733.019499:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019510:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019530:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.019538:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b79f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.019671:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507218688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31663us (32134us total) trans 4294971918 rc 0/0 00000100:00100000:1.0:1789760733.019682:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4628 00000100:00100000:1.0:1789760733.019728:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714256500153 time 714303294351 state 0x2 ref 0 00000100:00100000:2.0:1789760733.027889:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760733.029355:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218816 00000100:00100000:1.0:1789760733.029474:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218816 00000100:00100000:1.0:1789760733.029595:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714256500153 time 714303294351 state 0x0 ref 1 00000100:00100000:1.0:1789760733.029611:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714303554767 time 714303294351 state 0x1 ref 1 00000100:00100000:1.0:1789760733.029644:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714303554767 time 714344947020 state 0x0 ref 1 00000100:00100000:1.0:1789760733.029657:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507218816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760733.029720:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18837504, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0x9007 00010000:00010000:1.0:1789760733.029730:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.029747:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760733.029752:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.029759:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.029772:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760733.033366:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760733.086404:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760733.099908:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18837504 -> 18841600 00002000:00010000:3.0:1789760733.099918:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:3.0:1789760733.099920:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:3.0:1789760733.099933:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.099947:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.099956:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.099962:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760733.099967:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.099980:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.100004:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.100012:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b79fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.100111:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507218816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70452us (72227us total) trans 4294971919 rc 0/0 00000100:00100000:3.0:1789760733.100121:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4629 00000100:00100000:3.0:1789760733.100129:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714303554767 time 714344947020 state 0x2 ref 0 00000001:00080000:0.0:1789760733.103075:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971919 is committed 00000100:00100000:2.0:1789760733.111461:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760733.111600:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507218944 00000100:00100000:3.0:1789760733.111610:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507218944 00000100:00100000:3.0:1789760733.111662:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714303554767 time 714344947020 state 0x0 ref 1 00000100:00100000:3.0:1789760733.111677:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714345207436 time 714344947020 state 0x1 ref 1 00000100:00100000:3.0:1789760733.111738:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714345207436 time 714427040345 state 0x0 ref 1 00000100:00100000:3.0:1789760733.111751:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507218944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760733.111801:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18841600, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:3.0:1789760733.111810:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.111826:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760733.111830:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.111836:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.111859:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.153515:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18841600 -> 18845696 00010000:00010000:3.0:1789760733.153537:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153551:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153561:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153568:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760733.153573:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153583:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153603:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.153648:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.153802:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507218944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42047us (42345us total) trans 4294971920 rc 0/0 00000100:00100000:3.0:1789760733.153812:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4630 00000100:00100000:3.0:1789760733.153832:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714345207436 time 714427040345 state 0x2 ref 0 00000001:00080000:1.0:1789760733.155006:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971920 is committed 00000100:00100000:1.0:1789760733.160997:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.161185:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219072 00000100:00100000:2.0:1789760733.161195:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219072 00000100:00100000:2.0:1789760733.161258:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714345207436 time 714427040345 state 0x0 ref 1 00000100:00100000:2.0:1789760733.161273:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714427300761 time 714427040345 state 0x1 ref 1 00000100:00100000:2.0:1789760733.161309:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714427300761 time 714476612455 state 0x0 ref 1 00000100:00100000:2.0:1789760733.161321:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507219072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.161369:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18845696, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:2.0:1789760733.161378:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.161395:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.161401:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.161407:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.161421:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.200832:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18845696 -> 18849792 00000001:00080000:3.0:1789760733.200883:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971921 is committed 00010000:00010000:2.0:1789760733.200971:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.200982:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.200988:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.200992:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.200996:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.201002:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.201017:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.201022:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7a0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.201163:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507219072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39838us (40169us total) trans 4294971921 rc 0/0 00000100:00100000:2.0:1789760733.201174:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4631 00000100:00100000:2.0:1789760733.201183:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714427300761 time 714476612455 state 0x2 ref 0 00000100:00100000:3.0:1789760733.204234:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760733.204530:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219200 00000100:00100000:1.0:1789760733.204540:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219200 00000100:00100000:1.0:1789760733.204591:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714427300761 time 714476612455 state 0x0 ref 1 00000100:00100000:1.0:1789760733.204607:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714476872871 time 714476612455 state 0x1 ref 1 00000100:00100000:1.0:1789760733.204639:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714476872871 time 714519942181 state 0x0 ref 1 00000100:00100000:1.0:1789760733.204651:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507219200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760733.204720:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18849792, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:1.0:1789760733.204729:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.204743:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760733.204748:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.204754:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.204767:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.232545:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971922 is committed 00002000:00010000:2.0:1789760733.234945:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18849792 -> 18853888 00010000:00010000:2.0:1789760733.234963:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.234975:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.234983:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.234990:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.234994:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.235003:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.235021:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.235033:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7a11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.235134:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507219200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30480us (30904us total) trans 4294971922 rc 0/0 00000100:00100000:2.0:1789760733.235144:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4632 00000100:00100000:2.0:1789760733.235153:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714476872871 time 714519942181 state 0x2 ref 0 00000100:00100000:1.0:1789760733.239378:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760733.239479:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219328 00000100:00100000:3.0:1789760733.239491:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219328 00000100:00100000:3.0:1789760733.239544:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714476872871 time 714519942181 state 0x0 ref 1 00000100:00100000:3.0:1789760733.239558:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714520202597 time 714519942181 state 0x1 ref 1 00000100:00100000:3.0:1789760733.239586:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714520202597 time 714554889481 state 0x0 ref 1 00000100:00100000:3.0:1789760733.239599:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507219328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760733.239646:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18853888, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:3.0:1789760733.239656:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.239671:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760733.239676:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.239683:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.239740:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.277629:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971923 is committed 00002000:00010000:0.0:1789760733.277759:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18853888 -> 18857984 00010000:00010000:0.0:1789760733.277784:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277798:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277806:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277812:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760733.277817:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277830:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277858:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.277867:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7a18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.277970:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507219328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38370us (38601us total) trans 4294971923 rc 0/0 00000100:00100000:0.0:1789760733.277981:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4633 00000100:00100000:0.0:1789760733.277990:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714520202597 time 714554889481 state 0x2 ref 0 00000100:00100000:1.0:1789760733.281353:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.281473:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219456 00000100:00100000:2.0:1789760733.281485:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219456 00000100:00100000:2.0:1789760733.281543:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714520202597 time 714554889481 state 0x0 ref 1 00000100:00100000:2.0:1789760733.281559:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714555149897 time 714554889481 state 0x1 ref 1 00000100:00100000:2.0:1789760733.281629:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714555149897 time 714596931598 state 0x0 ref 1 00000100:00100000:2.0:1789760733.281641:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507219456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.281688:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18857984, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:2.0:1789760733.281739:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.281755:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.281759:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.281775:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.281790:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.305587:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971924 is committed 00002000:00010000:3.0:1789760733.305813:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18857984 -> 18862080 00010000:00010000:3.0:1789760733.305827:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.305840:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.305879:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.306078:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760733.306083:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.306092:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.306111:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.306118:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7a1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.306256:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507219456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24612us (24907us total) trans 4294971924 rc 0/0 00000100:00100000:3.0:1789760733.306267:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4634 00000100:00100000:3.0:1789760733.306276:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714555149897 time 714596931598 state 0x2 ref 0 00000100:00100000:3.0:1789760733.319925:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.320014:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219584 00000100:00100000:2.0:1789760733.320025:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219584 00000100:00100000:2.0:1789760733.320078:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714555149897 time 714596931598 state 0x0 ref 1 00000100:00100000:2.0:1789760733.320093:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714597192014 time 714596931598 state 0x1 ref 1 00000100:00100000:2.0:1789760733.320122:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714597192014 time 714635425594 state 0x0 ref 1 00000100:00100000:2.0:1789760733.320135:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507219584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.320193:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18862080, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:2.0:1789760733.320202:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.320218:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.320222:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.320229:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.320245:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.352365:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18862080 -> 18866176 00000001:00080000:0.0:1789760733.352379:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971925 is committed 00010000:00010000:2.0:1789760733.352387:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352400:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352411:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352419:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.352425:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352435:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352454:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.352463:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.352554:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507219584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32418us (32634us total) trans 4294971925 rc 0/0 00000100:00100000:2.0:1789760733.352565:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4635 00000100:00100000:2.0:1789760733.352576:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714597192014 time 714635425594 state 0x2 ref 0 00000100:00100000:1.0:1789760733.364561:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760733.364801:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219712 00000100:00100000:0.0:1789760733.364812:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219712 00000100:00100000:0.0:1789760733.364880:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714597192014 time 714635425594 state 0x0 ref 1 00000100:00100000:0.0:1789760733.364895:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714635686010 time 714635425594 state 0x1 ref 1 00000100:00100000:0.0:1789760733.364924:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714635686010 time 714680227488 state 0x0 ref 1 00000100:00100000:0.0:1789760733.364936:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507219712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760733.365024:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18866176, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:0.0:1789760733.365033:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.365048:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760733.365053:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.365058:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.365073:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.408131:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760733.408155:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538269056:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760733.408234:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760733.409009:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538269184:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.409051:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760733.409124:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269184 00000100:00100000:1.0:1789760733.409131:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269184 00000100:00100000:1.0:1789760733.409187:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760733.409196:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223496050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538269184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.409396:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223496050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538269184:12345-0@lo:400:kworker.0 Request processed in 200us (343us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.409404:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760733.409498:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538269440:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.409523:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760733.409586:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538269184:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.409602:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760733.409608:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538269312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.409632:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760733.409681:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760733.409687:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538269568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.409726:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.409918:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269056 00000100:00100000:0.0:1789760733.409923:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269056 00000100:00100000:0.0:1789760733.409962:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789760733.409969:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538269056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760733.409996:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760733.410033:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538269056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (1800us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.410040:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789760733.410058:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269440 00000100:00100000:0.0:1789760733.410061:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269440 00000100:00100000:0.0:1789760733.410085:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789760733.410091:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538269440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.410122:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538269440:12345-0@lo:400:kworker.0 Request processed in 31us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.410131:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789760733.410154:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538269696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.410249:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538269952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.410275:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.410418:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.410438:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538270208:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.410460:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.410512:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789760733.410518:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538269824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.410545:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269696 00000100:00100000:0.0:1789760733.410549:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269696 00000100:00100000:0.0:1789760733.410571:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1789760733.410577:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538269696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.410611:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538269696:12345-0@lo:400:kworker.0 Request processed in 34us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.410616:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1789760733.410633:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270208 00000100:00100000:0.0:1789760733.410635:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270208 00000100:00100000:0.0:1789760733.410660:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789760733.410667:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538270208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.410715:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538270208:12345-0@lo:400:kworker.0 Request processed in 48us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.410720:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:1.0:1789760733.410873:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538270464:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.410915:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760733.410980:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538270080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.411020:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269312 00000100:00100000:3.0:1789760733.411027:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269312 00000100:00100000:2.0:1789760733.411056:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760733.411069:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269568 00000100:00100000:3.0:1789760733.411075:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269568 00000100:00100000:2.0:1789760733.411099:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538270592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.411104:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269952 00000100:00100000:3.0:1789760733.411109:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269952 00000100:00100000:2.0:1789760733.411131:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760733.411138:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 872 00000100:00100000:3.0:1789760733.411151:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538269312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.411248:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538269312:12345-0@lo:400:kworker.0 Request processed in 103us (1615us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.411254:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 872 00000100:00100000:3.0:1789760733.411266:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 873 00000100:00100000:3.0:1789760733.411271:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538269568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.411301:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538269568:12345-0@lo:400:kworker.0 Request processed in 22us (1566us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.411308:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 873 00000100:00100000:3.0:1789760733.411324:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 874 00000100:00100000:3.0:1789760733.411331:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538269952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.411364:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538269952:12345-0@lo:400:kworker.0 Request processed in 33us (1088us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.411370:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 874 00000100:00080000:2.0:1789760733.411377:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760733.411386:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760733.411392:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760733.411453:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270080 00000100:00100000:2.0:1789760733.411461:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270080 00000100:00100000:2.0:1789760733.411521:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270592 00000100:00100000:2.0:1789760733.411524:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270592 00000100:00100000:2.0:1789760733.411562:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789760733.411575:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538270080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.411646:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538270080:12345-0@lo:400:kworker.0 Request processed in 75us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760733.411654:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789760733.411682:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789760733.411723:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538270592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.411763:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538270592:12345-0@lo:400:kworker.0 Request processed in 72us (633us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760733.411772:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00080000:2.0:1789760733.411811:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760733.411814:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760733.411870:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538270080:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.411886:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538270592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.411984:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538269568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.412001:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538269440:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.412021:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538269056:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760733.412413:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538269952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.412480:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270464 00000100:00100000:0.0:1789760733.412485:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270464 00000100:00080000:1.0:1789760733.412553:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760733.412591:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 875 00000100:00100000:3.0:1789760733.412600:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538270464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.412708:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538270464:12345-0@lo:400:kworker.0 Request processed in 94us (1777us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.412715:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 875 00000100:00100000:1.0:1789760733.412838:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538270464:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.412963:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.412996:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538270336:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.413026:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.413113:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538269696:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.413123:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538270208:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.413133:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538269312:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.413148:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538270720:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760733.413181:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.413211:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270720 00000100:00100000:0.0:1789760733.413215:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270720 00000100:00100000:0.0:1789760733.413240:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1789760733.413246:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538270720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760733.413260:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760733.413397:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538270720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (109us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.413406:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1789760733.413458:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538270720:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760733.413527:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538269824 00000100:00100000:1.0:1789760733.413533:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538269824 00000100:00100000:1.0:1789760733.413585:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270336 00000100:00100000:1.0:1789760733.413590:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270336 00000100:00100000:1.0:1789760733.413633:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 876 00000100:00100000:1.0:1789760733.413641:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538269824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.413686:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538269824:12345-0@lo:400:kworker.0 Request processed in 46us (722us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.413722:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 876 00000100:00100000:1.0:1789760733.413737:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 877 00000100:00100000:1.0:1789760733.413743:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538270336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.413771:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538270336:12345-0@lo:400:kworker.0 Request processed in 29us (745us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.413776:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 877 00000100:00080000:0.0:1789760733.413841:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760733.413872:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538269824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.413882:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538270336:0@lo:400:kworker.0 00000001:00080000:0.0:1789760733.417066:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971926 is committed 00002000:00010000:2.0:1789760733.417306:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18866176 -> 18870272 00010000:00010000:2.0:1789760733.417330:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417344:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417353:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417361:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.417366:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417375:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417397:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.417404:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7a2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.417527:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507219712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52585us (52968us total) trans 4294971926 rc 0/0 00000100:00100000:2.0:1789760733.417540:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4636 00000100:00100000:2.0:1789760733.417553:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714635686010 time 714680227488 state 0x2 ref 0 00000100:00100000:1.0:1789760733.423274:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760733.423328:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219840 00000100:00100000:3.0:1789760733.423338:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219840 00000100:00100000:3.0:1789760733.423389:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714635686010 time 714680227488 state 0x0 ref 1 00000100:00100000:3.0:1789760733.423403:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714680487904 time 714680227488 state 0x1 ref 1 00000100:00100000:3.0:1789760733.423435:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714680487904 time 714738737943 state 0x0 ref 1 00000100:00100000:3.0:1789760733.423447:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507219840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760733.423497:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18870272, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:3.0:1789760733.423506:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.423526:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760733.423531:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.423537:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.423550:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.472747:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971927 is committed 00002000:00010000:2.0:1789760733.472927:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18870272 -> 18874368 00010000:00010000:2.0:1789760733.472943:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.472955:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.472964:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.472971:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.472976:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.472987:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.473009:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.473016:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7a34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.473114:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507219840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49663us (49844us total) trans 4294971927 rc 0/0 00000100:00100000:2.0:1789760733.473124:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4637 00000100:00100000:2.0:1789760733.473133:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714680487904 time 714738737943 state 0x2 ref 0 00000100:00100000:0.0:1789760733.477120:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.478472:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507219968 00000100:00100000:2.0:1789760733.478489:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507219968 00000100:00100000:2.0:1789760733.478562:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714680487904 time 714738737943 state 0x0 ref 1 00000100:00100000:2.0:1789760733.478589:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714738998359 time 714738737943 state 0x1 ref 1 00000100:00100000:2.0:1789760733.478624:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714738998359 time 714793926935 state 0x0 ref 1 00000100:00100000:2.0:1789760733.478638:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507219968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.478684:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18874368, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9007 00010000:00010000:2.0:1789760733.478719:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.478734:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.478739:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.478745:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.478761:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.545356:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971928 is committed 00002000:00010000:2.0:1789760733.545528:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18874368 -> 18878464 00002000:00010000:2.0:1789760733.545533:0:8090:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:2.0:1789760733.545547:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545567:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545576:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545596:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.545601:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545611:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545631:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.545640:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7a3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.545769:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507219968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67128us (68660us total) trans 4294971928 rc 0/0 00000100:00100000:2.0:1789760733.545779:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4638 00000100:00100000:2.0:1789760733.545789:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714738998359 time 714793926935 state 0x2 ref 0 00000100:00100000:1.0:1789760733.549568:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.549744:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220096 00000100:00100000:2.0:1789760733.549754:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220096 00000100:00100000:2.0:1789760733.549810:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714738998359 time 714793926935 state 0x0 ref 1 00000100:00100000:2.0:1789760733.549825:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714794187351 time 714793926935 state 0x1 ref 1 00000100:00100000:2.0:1789760733.549860:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714794187351 time 714865163135 state 0x0 ref 1 00000100:00100000:2.0:1789760733.549872:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.551875:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18878464, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:2.0:1789760733.551898:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.551914:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.551919:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.551925:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.551941:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.613297:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971929 is committed 00002000:00010000:3.0:1789760733.613455:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18878464 -> 18882560 00010000:00010000:3.0:1789760733.613470:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613483:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613492:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613499:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760733.613504:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613513:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613539:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.613547:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7a42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.613648:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63773us (64085us total) trans 4294971929 rc 0/0 00000100:00100000:3.0:1789760733.613658:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4639 00000100:00100000:3.0:1789760733.613668:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714794187351 time 714865163135 state 0x2 ref 0 00000100:00100000:1.0:1789760733.618403:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760733.622102:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220224 00000100:00100000:3.0:1789760733.622116:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220224 00000100:00100000:2.0:1789760733.622333:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714794187351 time 714865163135 state 0x0 ref 1 00000100:00100000:2.0:1789760733.622349:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714865423551 time 714865163135 state 0x1 ref 1 00000100:00100000:2.0:1789760733.622378:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714865423551 time 714937681825 state 0x0 ref 1 00000100:00100000:2.0:1789760733.622394:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.622439:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18882560, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:2.0:1789760733.622447:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.622460:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760733.622465:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.622470:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.622483:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.710305:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971930 is committed 00002000:00010000:2.0:1789760733.710482:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18882560 -> 18886656 00010000:00010000:2.0:1789760733.710499:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710511:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710520:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710528:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.710532:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710542:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710562:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.710570:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7a49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.710941:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 88544us (92542us total) trans 4294971930 rc 0/0 00000100:00100000:2.0:1789760733.710952:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4640 00000100:00100000:2.0:1789760733.710961:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714865423551 time 714937681825 state 0x2 ref 0 00000100:00100000:2.0:1789760733.714525:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760733.717407:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220352 00000100:00100000:1.0:1789760733.717421:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220352 00000100:00100000:1.0:1789760733.717488:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714865423551 time 714937681825 state 0x0 ref 1 00000100:00100000:1.0:1789760733.717503:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714937942241 time 714937681825 state 0x1 ref 1 00000100:00100000:1.0:1789760733.717533:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714937942241 time 715032836677 state 0x0 ref 1 00000100:00100000:1.0:1789760733.717545:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507220352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760733.717601:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18886656, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:1.0:1789760733.717610:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.717625:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760733.717629:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.717635:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.717649:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.791420:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18886656 -> 18890752 00010000:00010000:3.0:1789760733.791440:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791452:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791462:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791470:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760733.791474:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791489:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791506:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.791513:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7a50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.791560:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971931 is committed 00000100:00100000:3.0:1789760733.791608:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507220352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 74060us (77086us total) trans 4294971931 rc 0/0 00000100:00100000:3.0:1789760733.791618:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4641 00000100:00100000:3.0:1789760733.791628:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714937942241 time 715032836677 state 0x2 ref 0 00000100:00100000:3.0:1789760733.795630:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760733.795735:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220480 00000100:00100000:2.0:1789760733.795745:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220480 00000100:00100000:2.0:1789760733.795799:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 714937942241 time 715032836677 state 0x0 ref 1 00000100:00100000:2.0:1789760733.795815:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715033097093 time 715032836677 state 0x1 ref 1 00000100:00100000:2.0:1789760733.795857:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715033097093 time 715111160012 state 0x0 ref 1 00000100:00100000:2.0:1789760733.795869:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507220480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.796088:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18890752, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:2.0:1789760733.796099:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.796116:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760733.796120:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.796126:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.796141:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760733.854180:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18890752 -> 18894848 00010000:00010000:0.0:1789760733.854198:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854216:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854226:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854233:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760733.854239:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854248:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854267:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.854276:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7a57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.854412:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507220480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58540us (58786us total) trans 4294971932 rc 0/0 00000100:00100000:0.0:1789760733.854423:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4642 00000100:00100000:0.0:1789760733.854433:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715033097093 time 715111160012 state 0x2 ref 0 00000001:00080000:0.0:1789760733.854471:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971932 is committed 00000100:00100000:2.0:1789760733.860617:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760733.860767:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220608 00000100:00100000:0.0:1789760733.860777:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220608 00000100:00100000:0.0:1789760733.860831:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715033097093 time 715111160012 state 0x0 ref 1 00000100:00100000:0.0:1789760733.860860:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715111420428 time 715111160012 state 0x1 ref 1 00000100:00100000:0.0:1789760733.860900:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715111420428 time 715176202172 state 0x0 ref 1 00000100:00100000:0.0:1789760733.860913:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760733.860968:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18894848, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:0.0:1789760733.860977:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.860995:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760733.861001:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.861008:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.861022:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.922183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971933 is committed 00002000:00010000:1.0:1789760733.922279:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18894848 -> 18898944 00010000:00010000:1.0:1789760733.922303:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922315:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922327:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922334:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760733.922339:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922348:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922366:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.922374:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7a5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.922466:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61551us (61852us total) trans 4294971933 rc 0/0 00000100:00100000:1.0:1789760733.922477:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4643 00000100:00100000:1.0:1789760733.922486:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715111420428 time 715176202172 state 0x2 ref 0 00000100:00100000:2.0:1789760733.926633:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760733.927027:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220736 00000100:00100000:3.0:1789760733.927037:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220736 00000100:00100000:2.0:1789760733.927189:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715111420428 time 715176202172 state 0x0 ref 1 00000100:00100000:2.0:1789760733.927480:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715176462588 time 715176202172 state 0x1 ref 1 00000100:00100000:2.0:1789760733.927553:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715176462588 time 715242854757 state 0x0 ref 1 00000100:00100000:2.0:1789760733.927568:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760733.927619:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18898944, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:2.0:1789760733.927628:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.927643:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760733.927648:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.927656:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.927670:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.958981:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971934 is committed 00002000:00010000:2.0:1789760733.959119:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18898944 -> 18903040 00010000:00010000:2.0:1789760733.959134:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959146:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959155:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959163:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760733.959168:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959179:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959200:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.959208:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7a65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.959398:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31827us (32770us total) trans 4294971934 rc 0/0 00000100:00100000:2.0:1789760733.959410:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4644 00000100:00100000:2.0:1789760733.959419:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715176462588 time 715242854757 state 0x2 ref 0 00000100:00100000:0.0:1789760733.963654:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760733.963888:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220864 00000100:00100000:1.0:1789760733.963898:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220864 00000100:00100000:1.0:1789760733.963950:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715176462588 time 715242854757 state 0x0 ref 1 00000100:00100000:1.0:1789760733.963967:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715243115173 time 715242854757 state 0x1 ref 1 00000100:00100000:1.0:1789760733.964033:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715243115173 time 715279335495 state 0x0 ref 1 00000100:00100000:1.0:1789760733.964047:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760733.964097:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18903040, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:1.0:1789760733.964105:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.964121:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760733.964125:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.964131:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.964145:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.990583:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971935 is committed 00002000:00010000:0.0:1789760733.990660:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18903040 -> 18907136 00010000:00010000:0.0:1789760733.990675:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990687:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990765:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990774:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760733.990779:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990789:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990808:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.990817:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7a6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.991015:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507220864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26965us (27364us total) trans 4294971935 rc 0/0 00000100:00100000:0.0:1789760733.991025:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4645 00000100:00100000:0.0:1789760733.991035:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715243115173 time 715279335495 state 0x2 ref 0 00000100:00100000:2.0:1789760733.994164:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760733.994256:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507220992 00000100:00100000:1.0:1789760733.994266:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507220992 00000100:00100000:1.0:1789760733.994381:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715243115173 time 715279335495 state 0x0 ref 1 00000100:00100000:1.0:1789760733.994397:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715279595911 time 715279335495 state 0x1 ref 1 00000100:00100000:1.0:1789760733.994460:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715279595911 time 715309762746 state 0x0 ref 1 00000100:00100000:1.0:1789760733.994472:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507220992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760733.994518:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18907136, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0x9807 00010000:00010000:1.0:1789760733.994527:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.994544:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760733.994549:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.994555:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.994570:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.026489:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971936 is committed 00002000:00010000:2.0:1789760734.026586:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18907136 -> 18911232 00002000:00010000:2.0:1789760734.026591:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:2.0:1789760734.026594:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:2.0:1789760734.026606:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026619:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026628:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026635:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.026640:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026648:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026672:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.026680:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b7a73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.026818:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507220992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32343us (32660us total) trans 4294971936 rc 0/0 00000100:00100000:2.0:1789760734.026828:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4646 00000100:00100000:2.0:1789760734.026838:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715279595911 time 715309762746 state 0x2 ref 0 00000100:00100000:1.0:1789760734.031181:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.031262:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221120 00000100:00100000:3.0:1789760734.031272:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221120 00000100:00100000:3.0:1789760734.031343:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715279595911 time 715309762746 state 0x0 ref 1 00000100:00100000:3.0:1789760734.031359:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715310023162 time 715309762746 state 0x1 ref 1 00000100:00100000:3.0:1789760734.031390:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715310023162 time 715346693833 state 0x0 ref 1 00000100:00100000:3.0:1789760734.031406:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.031455:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18911232, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.031464:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.031480:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760734.031484:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.031490:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.031505:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.057263:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18911232 -> 18915328 00010000:00010000:3.0:1789760734.057283:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057363:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057373:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057380:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760734.057384:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057393:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057413:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.057419:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7a7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.057517:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507221120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26108us (26342us total) trans 4294971937 rc 0/0 00000100:00100000:3.0:1789760734.057527:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4647 00000100:00100000:3.0:1789760734.057536:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715310023162 time 715346693833 state 0x2 ref 0 00000001:00080000:3.0:1789760734.057575:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971937 is committed 00000100:00100000:2.0:1789760734.060505:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.060574:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221248 00000100:00100000:3.0:1789760734.060583:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221248 00000100:00100000:3.0:1789760734.060636:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715310023162 time 715346693833 state 0x0 ref 1 00000100:00100000:3.0:1789760734.060651:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715346954249 time 715346693833 state 0x1 ref 1 00000100:00100000:3.0:1789760734.060681:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715346954249 time 715375983945 state 0x0 ref 1 00000100:00100000:3.0:1789760734.060708:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.060755:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18915328, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.060765:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.060779:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760734.060783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.060789:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.060803:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.092444:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971938 is committed 00002000:00010000:1.0:1789760734.092550:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18915328 -> 18919424 00010000:00010000:1.0:1789760734.092566:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092579:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092588:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092597:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760734.092602:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092616:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092636:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.092644:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7a81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.092819:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32114us (32308us total) trans 4294971938 rc 0/0 00000100:00100000:1.0:1789760734.092830:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4648 00000100:00100000:1.0:1789760734.092840:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715346954249 time 715375983945 state 0x2 ref 0 00000100:00100000:0.0:1789760734.097058:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760734.097202:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221376 00000100:00100000:1.0:1789760734.097211:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221376 00000100:00100000:1.0:1789760734.097268:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715346954249 time 715375983945 state 0x0 ref 1 00000100:00100000:1.0:1789760734.097285:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715376244361 time 715375983945 state 0x1 ref 1 00000100:00100000:1.0:1789760734.097407:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715376244361 time 715412709845 state 0x0 ref 1 00000100:00100000:1.0:1789760734.097421:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760734.097470:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18919424, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:1.0:1789760734.097479:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.097493:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760734.097497:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.097503:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.097516:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.109801:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760734.132473:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971939 is committed 00002000:00010000:2.0:1789760734.132569:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18919424 -> 18923520 00010000:00010000:2.0:1789760734.132584:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132598:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132607:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132615:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.132620:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132629:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132649:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.132656:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7a88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.132789:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35366us (35735us total) trans 4294971939 rc 0/0 00000100:00100000:2.0:1789760734.132798:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4649 00000100:00100000:2.0:1789760734.132806:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715376244361 time 715412709845 state 0x2 ref 0 00000100:00100000:2.0:1789760734.136370:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.136494:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221504 00000100:00100000:3.0:1789760734.136502:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221504 00000100:00100000:3.0:1789760734.136550:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715376244361 time 715412709845 state 0x0 ref 1 00000100:00100000:3.0:1789760734.136565:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715412970261 time 715412709845 state 0x1 ref 1 00000100:00100000:3.0:1789760734.136595:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715412970261 time 715451898184 state 0x0 ref 1 00000100:00100000:3.0:1789760734.136607:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.136649:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18923520, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.136658:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.136673:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760734.136678:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.136684:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.136722:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.177550:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971940 is committed 00002000:00010000:2.0:1789760734.177771:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18923520 -> 18927616 00010000:00010000:2.0:1789760734.177787:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.177800:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.178161:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.178175:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.178180:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.178190:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.178305:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.178316:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7a8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.178422:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41811us (42056us total) trans 4294971940 rc 0/0 00000100:00100000:2.0:1789760734.178432:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4650 00000100:00100000:2.0:1789760734.178442:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715412970261 time 715451898184 state 0x2 ref 0 00000100:00100000:3.0:1789760734.181984:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760734.182109:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221632 00000100:00100000:1.0:1789760734.182120:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221632 00000100:00100000:1.0:1789760734.182173:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715412970261 time 715451898184 state 0x0 ref 1 00000100:00100000:1.0:1789760734.182188:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715452158600 time 715451898184 state 0x1 ref 1 00000100:00100000:1.0:1789760734.183040:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715452158600 time 715498340026 state 0x0 ref 1 00000100:00100000:1.0:1789760734.183055:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760734.183102:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18927616, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:1.0:1789760734.183110:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.183125:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760734.183130:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.183135:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.183154:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.215615:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971941 is committed 00002000:00010000:2.0:1789760734.215688:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18927616 -> 18931712 00010000:00010000:2.0:1789760734.215735:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215743:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215753:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215758:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.215761:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215771:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215790:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.215797:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7a96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.215905:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32847us (33923us total) trans 4294971941 rc 0/0 00000100:00100000:2.0:1789760734.215914:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4651 00000100:00100000:2.0:1789760734.215923:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715452158600 time 715498340026 state 0x2 ref 0 00000100:00100000:0.0:1789760734.224947:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.226063:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221760 00000100:00100000:2.0:1789760734.226074:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221760 00000100:00100000:2.0:1789760734.226148:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715452158600 time 715498340026 state 0x0 ref 1 00000100:00100000:2.0:1789760734.226164:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715498600442 time 715498340026 state 0x1 ref 1 00000100:00100000:2.0:1789760734.226202:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715498600442 time 715541505186 state 0x0 ref 1 00000100:00100000:2.0:1789760734.226215:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.226273:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18931712, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.226282:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.226298:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760734.226302:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.226308:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.226323:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.261839:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18931712 -> 18935808 00010000:00010000:2.0:1789760734.261879:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261898:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261907:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261914:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.261921:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261930:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261950:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.261957:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7a9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.261960:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971942 is committed 00000100:00100000:2.0:1789760734.262299:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507221760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36081us (37356us total) trans 4294971942 rc 0/0 00000100:00100000:2.0:1789760734.262309:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4652 00000100:00100000:2.0:1789760734.262318:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715498600442 time 715541505186 state 0x2 ref 0 00000100:00100000:3.0:1789760734.265395:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.265598:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507221888 00000100:00100000:2.0:1789760734.265609:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507221888 00000100:00100000:2.0:1789760734.265662:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715498600442 time 715541505186 state 0x0 ref 1 00000100:00100000:2.0:1789760734.265675:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715541765602 time 715541505186 state 0x1 ref 1 00000100:00100000:2.0:1789760734.266063:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715541765602 time 715581363863 state 0x0 ref 1 00000100:00100000:3.0:1789760734.266250:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.266311:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18935808, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.266321:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.266337:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760734.266342:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.266349:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.266363:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.307253:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971943 is committed 00002000:00010000:0.0:1789760734.307541:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18935808 -> 18939904 00010000:00010000:0.0:1789760734.307558:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307572:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307581:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307588:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760734.307594:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307603:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307622:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.307630:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7aa4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.307754:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507221888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41675us (42366us total) trans 4294971943 rc 0/0 00000100:00100000:0.0:1789760734.307765:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4653 00000100:00100000:0.0:1789760734.307775:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715541765602 time 715581363863 state 0x2 ref 0 00000100:00100000:3.0:1789760734.314156:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.314258:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222016 00000100:00100000:2.0:1789760734.314267:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222016 00000100:00100000:2.0:1789760734.314318:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715541765602 time 715581363863 state 0x0 ref 1 00000100:00100000:2.0:1789760734.314332:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715581624279 time 715581363863 state 0x1 ref 1 00000100:00100000:2.0:1789760734.314362:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715581624279 time 715629665365 state 0x0 ref 1 00000100:00100000:2.0:1789760734.314374:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507222016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.314422:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18939904, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.314431:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.314447:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760734.314452:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.314458:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.314471:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.340194:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18939904 -> 18944000 00010000:00010000:2.0:1789760734.340216:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340229:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340239:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340247:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.340252:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340262:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340282:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.340291:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7aab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.340421:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507222016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26044us (26270us total) trans 4294971944 rc 0/0 00000100:00100000:2.0:1789760734.340432:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4654 00000100:00100000:2.0:1789760734.340442:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715581624279 time 715629665365 state 0x2 ref 0 00000001:00080000:0.0:1789760734.340500:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971944 is committed 00000100:00100000:2.0:1789760734.343860:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760734.344003:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222144 00000100:00100000:1.0:1789760734.344012:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222144 00000100:00100000:1.0:1789760734.344062:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715581624279 time 715629665365 state 0x0 ref 1 00000100:00100000:1.0:1789760734.344077:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715629925781 time 715629665365 state 0x1 ref 1 00000100:00100000:1.0:1789760734.344146:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715629925781 time 715659448523 state 0x0 ref 1 00000100:00100000:1.0:1789760734.344157:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760734.344202:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18944000, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:1.0:1789760734.344210:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.344225:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760734.344229:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.344236:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.344249:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.371385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971945 is committed 00002000:00010000:2.0:1789760734.371585:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18944000 -> 18948096 00010000:00010000:2.0:1789760734.371600:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371612:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371627:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371634:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.371639:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371648:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371668:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.371676:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7ab2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.371803:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27641us (27956us total) trans 4294971945 rc 0/0 00000100:00100000:2.0:1789760734.371813:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4655 00000100:00100000:2.0:1789760734.371823:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715629925781 time 715659448523 state 0x2 ref 0 00000100:00100000:3.0:1789760734.374792:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760734.375055:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222272 00000100:00100000:0.0:1789760734.375066:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222272 00000100:00100000:0.0:1789760734.375121:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715629925781 time 715659448523 state 0x0 ref 1 00000100:00100000:0.0:1789760734.375140:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715659708939 time 715659448523 state 0x1 ref 1 00000100:00100000:0.0:1789760734.375205:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715659708939 time 715690508684 state 0x0 ref 1 00000100:00100000:0.0:1789760734.375217:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507222272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760734.375263:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18948096, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:0.0:1789760734.375273:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.375297:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760734.375301:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.375307:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.375320:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.419758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971946 is committed 00002000:00010000:1.0:1789760734.419946:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18948096 -> 18952192 00010000:00010000:1.0:1789760734.419963:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.419976:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.419986:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.419994:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760734.419999:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.420009:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.420030:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.420038:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7ab9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.420133:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507222272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44912us (45346us total) trans 4294971946 rc 0/0 00000100:00100000:1.0:1789760734.420477:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4656 00000100:00100000:1.0:1789760734.420489:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715659708939 time 715690508684 state 0x2 ref 0 00000100:00100000:0.0:1789760734.428013:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760734.428036:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538270848:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760734.428102:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760734.428801:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760734.428815:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538270976:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760734.428862:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760734.429039:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270848 00000100:00100000:3.0:1789760734.429048:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270848 00000100:00100000:3.0:1789760734.429087:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538270976 00000100:00100000:3.0:1789760734.429090:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538270976 00000100:00100000:3.0:1789760734.429106:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789760734.429114:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538270848:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760734.429136:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3839885312 avail 3802136576 00000100:00100000:3.0:1789760734.429196:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538270848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (1094us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760734.429205:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789760734.429222:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1789760734.429228:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538270976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760734.429236:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3839885312 avail 3802136576 00000100:00100000:3.0:1789760734.429260:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538270976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 32us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760734.429266:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1789760734.429332:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.429380:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538270848:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760734.431010:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222400 00000100:00100000:3.0:1789760734.431020:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222400 00000100:00100000:3.0:1789760734.431064:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715659708939 time 715690508684 state 0x0 ref 1 00000100:00100000:3.0:1789760734.431078:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715690769100 time 715690508684 state 0x1 ref 1 00000100:00100000:2.0:1789760734.431227:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538270976:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760734.431338:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715690769100 time 715746639247 state 0x0 ref 1 00000100:00100000:3.0:1789760734.431350:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507222400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.431390:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18952192, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.431399:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.431414:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760734.431419:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.431425:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.431438:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.499482:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971947 is committed 00002000:00010000:2.0:1789760734.499687:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18952192 -> 18956288 00010000:00010000:2.0:1789760734.499728:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499740:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499749:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499757:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.499762:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499771:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499792:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.499800:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7ac0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.499921:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507222400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68567us (70596us total) trans 4294971947 rc 0/0 00000100:00100000:2.0:1789760734.499931:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4657 00000100:00100000:2.0:1789760734.499942:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715690769100 time 715746639247 state 0x2 ref 0 00000100:00100000:2.0:1789760734.503250:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760734.503614:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222528 00000100:00100000:1.0:1789760734.503624:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222528 00000100:00100000:1.0:1789760734.503682:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715690769100 time 715746639247 state 0x0 ref 1 00000100:00100000:1.0:1789760734.503717:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715746899663 time 715746639247 state 0x1 ref 1 00000100:00100000:1.0:1789760734.503746:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715746899663 time 715819050210 state 0x0 ref 1 00000100:00100000:1.0:1789760734.503758:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507222528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760734.503805:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18956288, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:1.0:1789760734.503814:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.503829:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760734.503834:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.503854:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.503869:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.546428:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971948 is committed 00002000:00010000:1.0:1789760734.546581:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18956288 -> 18960384 00010000:00010000:1.0:1789760734.546612:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546624:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546633:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546640:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760734.546646:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546654:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546674:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.546682:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ac7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.547025:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507222528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43262us (43778us total) trans 4294971948 rc 0/0 00000100:00100000:2.0:1789760734.547036:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4658 00000100:00100000:2.0:1789760734.547047:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715746899663 time 715819050210 state 0x2 ref 0 00000100:00100000:0.0:1789760734.552597:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760734.554628:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222656 00000100:00100000:1.0:1789760734.554642:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222656 00000100:00100000:1.0:1789760734.554739:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715746899663 time 715819050210 state 0x0 ref 1 00000100:00100000:1.0:1789760734.554755:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715819310626 time 715819050210 state 0x1 ref 1 00000100:00100000:1.0:1789760734.554777:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715819310626 time 715870080654 state 0x0 ref 1 00000100:00100000:1.0:1789760734.554792:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507222656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760734.554849:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18960384, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:1.0:1789760734.554858:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.554874:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760734.554879:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.554885:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.554898:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.623385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971949 is committed 00002000:00010000:1.0:1789760734.623464:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18960384 -> 18964480 00010000:00010000:1.0:1789760734.623479:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623492:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623502:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623510:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760734.623514:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623523:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623542:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.623549:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ace lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.623669:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507222656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68877us (71078us total) trans 4294971949 rc 0/0 00000100:00100000:1.0:1789760734.623681:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4659 00000100:00100000:1.0:1789760734.623690:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715819310626 time 715870080654 state 0x2 ref 0 00000100:00100000:3.0:1789760734.634525:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.634798:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222784 00000100:00100000:2.0:1789760734.634807:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222784 00000100:00100000:2.0:1789760734.634873:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715819310626 time 715870080654 state 0x0 ref 1 00000100:00100000:2.0:1789760734.634897:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715870341070 time 715870080654 state 0x1 ref 1 00000100:00100000:2.0:1789760734.634927:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715870341070 time 715950230437 state 0x0 ref 1 00000100:00100000:2.0:1789760734.634941:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.634988:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18964480, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.634998:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.635014:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760734.635018:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.635024:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.635042:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.684560:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971950 is committed 00002000:00010000:2.0:1789760734.684652:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18964480 -> 18968576 00010000:00010000:2.0:1789760734.684666:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684679:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684689:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684722:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.684728:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684737:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684756:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.684763:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ad5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.684857:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49913us (50338us total) trans 4294971950 rc 0/0 00000100:00100000:2.0:1789760734.684867:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4660 00000100:00100000:2.0:1789760734.684886:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715870341070 time 715950230437 state 0x2 ref 0 00000100:00100000:3.0:1789760734.691831:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.691935:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507222912 00000100:00100000:2.0:1789760734.691946:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507222912 00000100:00100000:2.0:1789760734.692001:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715870341070 time 715950230437 state 0x0 ref 1 00000100:00100000:2.0:1789760734.692016:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715950490853 time 715950230437 state 0x1 ref 1 00000100:00100000:2.0:1789760734.692044:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715950490853 time 716007347313 state 0x0 ref 1 00000100:00100000:2.0:1789760734.692057:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.692104:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18968576, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.692113:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.692129:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760734.692134:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.692139:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.692155:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.734178:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971951 is committed 00002000:00010000:2.0:1789760734.734588:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18968576 -> 18972672 00010000:00010000:2.0:1789760734.734604:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734617:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734628:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734635:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.734640:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734649:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734668:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.734676:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7adc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.734911:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507222912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42849us (43083us total) trans 4294971951 rc 0/0 00000100:00100000:2.0:1789760734.734925:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4661 00000100:00100000:2.0:1789760734.734934:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715950490853 time 716007347313 state 0x2 ref 0 00000100:00100000:3.0:1789760734.737910:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760734.738143:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223040 00000100:00100000:0.0:1789760734.738153:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223040 00000100:00100000:0.0:1789760734.738204:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 715950490853 time 716007347313 state 0x0 ref 1 00000100:00100000:0.0:1789760734.738219:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716007607729 time 716007347313 state 0x1 ref 1 00000100:00100000:0.0:1789760734.738246:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716007607729 time 716053549562 state 0x0 ref 1 00000100:00100000:0.0:1789760734.738258:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760734.738316:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18972672, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:0.0:1789760734.738326:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.738341:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760734.738346:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.738352:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.738365:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.768177:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971952 is committed 00002000:00010000:3.0:1789760734.768295:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18972672 -> 18976768 00010000:00010000:3.0:1789760734.768312:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768325:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768333:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768344:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760734.768348:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768356:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768374:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.768381:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7ae3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.768475:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30214us (30568us total) trans 4294971952 rc 0/0 00000100:00100000:3.0:1789760734.768486:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4662 00000100:00100000:3.0:1789760734.768496:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716007607729 time 716053549562 state 0x2 ref 0 00000100:00100000:0.0:1789760734.773685:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.773962:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223168 00000100:00100000:2.0:1789760734.773972:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223168 00000100:00100000:2.0:1789760734.774024:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716007607729 time 716053549562 state 0x0 ref 1 00000100:00100000:2.0:1789760734.774040:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716053809978 time 716053549562 state 0x1 ref 1 00000100:00100000:2.0:1789760734.774108:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716053809978 time 716089411004 state 0x0 ref 1 00000100:00100000:2.0:1789760734.774122:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.774169:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18976768, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.774178:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.774195:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760734.774200:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.774206:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.774220:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.803002:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18976768 -> 18980864 00010000:00010000:3.0:1789760734.803021:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803035:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803050:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803058:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760734.803064:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803074:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803094:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.803103:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7aea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.803198:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507223168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29074us (29518us total) trans 4294971953 rc 0/0 00000100:00100000:3.0:1789760734.803207:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4663 00000100:00100000:3.0:1789760734.803216:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716053809978 time 716089411004 state 0x2 ref 0 00000001:00080000:0.0:1789760734.803300:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971953 is committed 00000100:00100000:0.0:1789760734.806396:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.807806:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223296 00000100:00100000:2.0:1789760734.807819:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223296 00000100:00100000:3.0:1789760734.808073:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716053809978 time 716089411004 state 0x0 ref 1 00000100:00100000:3.0:1789760734.808089:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716089671420 time 716089411004 state 0x1 ref 1 00000100:00100000:3.0:1789760734.808160:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716089671420 time 716123463284 state 0x0 ref 1 00000100:00100000:3.0:1789760734.808182:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.808225:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18980864, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.808234:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.808249:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760734.808253:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.808260:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.808279:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.843189:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971954 is committed 00002000:00010000:2.0:1789760734.843413:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18980864 -> 18984960 00010000:00010000:2.0:1789760734.843432:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843446:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843454:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843461:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.843465:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843473:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843492:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.843499:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7af1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.843586:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35410us (37192us total) trans 4294971954 rc 0/0 00000100:00100000:2.0:1789760734.843595:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4664 00000100:00100000:2.0:1789760734.843605:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716089671420 time 716123463284 state 0x2 ref 0 00000100:00100000:1.0:1789760734.852621:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.852845:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223424 00000100:00100000:3.0:1789760734.852854:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223424 00000100:00100000:3.0:1789760734.852916:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716089671420 time 716123463284 state 0x0 ref 1 00000100:00100000:3.0:1789760734.852931:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716123723700 time 716123463284 state 0x1 ref 1 00000100:00100000:3.0:1789760734.852962:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716123723700 time 716168265208 state 0x0 ref 1 00000100:00100000:3.0:1789760734.852974:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.853026:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18984960, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.853037:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.853054:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760734.853058:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.853065:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.853078:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.892719:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971955 is committed 00002000:00010000:1.0:1789760734.893629:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18984960 -> 18989056 00010000:00010000:1.0:1789760734.893647:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893660:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893670:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893677:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760734.893682:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893717:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893801:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.893811:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7af8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.893921:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40943us (41303us total) trans 4294971955 rc 0/0 00000100:00100000:1.0:1789760734.893931:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4665 00000100:00100000:1.0:1789760734.893940:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716123723700 time 716168265208 state 0x2 ref 0 00000100:00100000:0.0:1789760734.898165:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760734.898218:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223552 00000100:00100000:3.0:1789760734.898228:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223552 00000100:00100000:3.0:1789760734.898329:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716123723700 time 716168265208 state 0x0 ref 1 00000100:00100000:3.0:1789760734.898347:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716168525624 time 716168265208 state 0x1 ref 1 00000100:00100000:3.0:1789760734.898431:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716168525624 time 716213732932 state 0x0 ref 1 00000100:00100000:3.0:1789760734.898445:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760734.898496:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18989056, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760734.898505:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.898521:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760734.898526:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.898532:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.898547:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.930159:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971956 is committed 00002000:00010000:2.0:1789760734.930333:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18989056 -> 18993152 00010000:00010000:2.0:1789760734.930348:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930361:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930371:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930378:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760734.930383:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930393:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930411:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.930419:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7aff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.930515:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32068us (32355us total) trans 4294971956 rc 0/0 00000100:00100000:2.0:1789760734.930528:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4666 00000100:00100000:2.0:1789760734.930538:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716168525624 time 716213732932 state 0x2 ref 0 00000100:00100000:0.0:1789760734.936615:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760734.936855:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223680 00000100:00100000:2.0:1789760734.936865:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223680 00000100:00100000:2.0:1789760734.936921:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716168525624 time 716213732932 state 0x0 ref 1 00000100:00100000:2.0:1789760734.936935:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716213993348 time 716213732932 state 0x1 ref 1 00000100:00100000:2.0:1789760734.936961:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716213993348 time 716252264558 state 0x0 ref 1 00000100:00100000:2.0:1789760734.936974:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760734.937024:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18993152, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:2.0:1789760734.937032:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.937046:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760734.937050:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.937068:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.937081:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.942013:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760734.942035:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538271104:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760734.942141:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760734.942216:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271104 00000100:00100000:2.0:1789760734.942225:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271104 00000100:00100000:2.0:1789760734.942268:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:2.0:1789760734.942276:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538271104:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760734.942322:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760734.942369:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538271104:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760734.942376:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1789760734.944468:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538271104:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760735.003586:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971957 is committed 00002000:00010000:2.0:1789760735.003782:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18993152 -> 18997248 00010000:00010000:2.0:1789760735.003798:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003810:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003819:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003826:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.003831:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003848:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003868:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003943:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.004046:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67070us (67436us total) trans 4294971957 rc 0/0 00000100:00100000:2.0:1789760735.004056:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4667 00000100:00100000:2.0:1789760735.004065:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716213993348 time 716252264558 state 0x2 ref 0 00000100:00100000:1.0:1789760735.012414:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760735.012531:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223808 00000100:00100000:3.0:1789760735.012542:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223808 00000100:00100000:3.0:1789760735.012616:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716213993348 time 716252264558 state 0x0 ref 1 00000100:00100000:3.0:1789760735.012632:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716252524974 time 716252264558 state 0x1 ref 1 00000100:00100000:3.0:1789760735.012663:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716252524974 time 716327965650 state 0x0 ref 1 00000100:00100000:3.0:1789760735.012676:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760735.012765:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18997248, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0x9807 00010000:00010000:3.0:1789760735.012774:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.012789:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760735.012793:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.012800:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.012814:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760735.013216:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760735.054585:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971958 is committed 00002000:00010000:2.0:1789760735.054776:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18997248 -> 19001344 00002000:00010000:2.0:1789760735.054781:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:2.0:1789760735.054784:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:2.0:1789760735.054796:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054808:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054818:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054825:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.054829:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054849:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054870:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054879:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7b0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.054970:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507223808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42292us (42563us total) trans 4294971958 rc 0/0 00000100:00100000:2.0:1789760735.054981:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4668 00000100:00100000:2.0:1789760735.054991:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716252524974 time 716327965650 state 0x2 ref 0 00000100:00100000:0.0:1789760735.059163:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.059253:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507223936 00000100:00100000:2.0:1789760735.059263:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507223936 00000100:00100000:2.0:1789760735.059311:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716252524974 time 716327965650 state 0x0 ref 1 00000100:00100000:2.0:1789760735.059325:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716328226066 time 716327965650 state 0x1 ref 1 00000100:00100000:2.0:1789760735.059355:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716328226066 time 716374659104 state 0x0 ref 1 00000100:00100000:2.0:1789760735.059368:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.059415:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19001344, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.059424:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.059439:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.059443:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.059450:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.059469:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.100147:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971959 is committed 00002000:00010000:2.0:1789760735.100435:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19001344 -> 19005440 00010000:00010000:2.0:1789760735.100451:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100462:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100471:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100478:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.100483:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100492:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100509:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.100516:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.100669:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507223936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41298us (41510us total) trans 4294971959 rc 0/0 00000100:00100000:2.0:1789760735.100679:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4669 00000100:00100000:2.0:1789760735.100689:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716328226066 time 716374659104 state 0x2 ref 0 00000100:00100000:0.0:1789760735.105966:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.106085:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224064 00000100:00100000:2.0:1789760735.106095:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224064 00000100:00100000:2.0:1789760735.106150:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716328226066 time 716374659104 state 0x0 ref 1 00000100:00100000:2.0:1789760735.106165:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716374919520 time 716374659104 state 0x1 ref 1 00000100:00100000:2.0:1789760735.106201:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716374919520 time 716421504284 state 0x0 ref 1 00000100:00100000:2.0:1789760735.106213:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507224064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.106260:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19005440, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.106270:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.106296:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.106301:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.106307:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.106322:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.132022:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760735.135880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971960 is committed 00002000:00010000:3.0:1789760735.136033:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19005440 -> 19009536 00010000:00010000:3.0:1789760735.136048:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136061:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136070:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136078:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760735.136083:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136092:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136110:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.136117:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7b1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.136301:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507224064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30085us (30341us total) trans 4294971960 rc 0/0 00000100:00100000:3.0:1789760735.136312:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4670 00000100:00100000:3.0:1789760735.136321:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716374919520 time 716421504284 state 0x2 ref 0 00000100:00100000:1.0:1789760735.140254:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.140410:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224192 00000100:00100000:2.0:1789760735.140421:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224192 00000100:00100000:2.0:1789760735.140482:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716374919520 time 716421504284 state 0x0 ref 1 00000100:00100000:2.0:1789760735.140498:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716421764700 time 716421504284 state 0x1 ref 1 00000100:00100000:2.0:1789760735.140529:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716421764700 time 716455832168 state 0x0 ref 1 00000100:00100000:2.0:1789760735.140541:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.140588:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19009536, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.140602:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.140620:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.140625:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.140632:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.140646:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.183732:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971961 is committed 00002000:00010000:2.0:1789760735.183940:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19009536 -> 19013632 00010000:00010000:2.0:1789760735.183956:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.183975:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.183985:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.183993:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.183998:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.184008:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.184029:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.184037:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.184167:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43622us (43917us total) trans 4294971961 rc 0/0 00000100:00100000:2.0:1789760735.184178:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4671 00000100:00100000:2.0:1789760735.184196:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716421764700 time 716455832168 state 0x2 ref 0 00000100:00100000:0.0:1789760735.195046:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.196965:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224320 00000100:00100000:2.0:1789760735.196981:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224320 00000100:00100000:0.0:1789760735.199757:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716421764700 time 716455832168 state 0x0 ref 1 00000100:00100000:0.0:1789760735.199777:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716456092584 time 716455832168 state 0x1 ref 1 00000100:00100000:0.0:1789760735.199880:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716456092584 time 716515175912 state 0x0 ref 1 00000100:00100000:0.0:1789760735.199895:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760735.199943:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19013632, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760735.199953:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.199969:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760735.199974:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.199981:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.199996:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.253793:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971962 is committed 00002000:00010000:0.0:1789760735.253918:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19013632 -> 19017728 00010000:00010000:0.0:1789760735.253933:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.253945:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.253956:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.253963:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.253967:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.253976:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.253995:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.254002:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7b29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.254093:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54197us (59052us total) trans 4294971962 rc 0/0 00000100:00100000:0.0:1789760735.254107:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4672 00000100:00100000:0.0:1789760735.254116:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716456092584 time 716515175912 state 0x2 ref 0 00000100:00100000:1.0:1789760735.258100:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760735.258197:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224448 00000100:00100000:3.0:1789760735.258207:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224448 00000100:00100000:3.0:1789760735.258262:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716456092584 time 716515175912 state 0x0 ref 1 00000100:00100000:3.0:1789760735.258277:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716515436328 time 716515175912 state 0x1 ref 1 00000100:00100000:3.0:1789760735.258403:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716515436328 time 716573704437 state 0x0 ref 1 00000100:00100000:3.0:1789760735.258416:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760735.258467:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19017728, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:3.0:1789760735.258476:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.258492:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760735.258496:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.258502:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.258516:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.312578:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19017728 -> 19021824 00000001:00080000:3.0:1789760735.312586:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971963 is committed 00010000:00010000:2.0:1789760735.312596:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312609:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312617:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312624:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.312629:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312638:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312657:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.312665:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7b30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19017728->19021823] (req 19017728->19021823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.312847:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54416us (54740us total) trans 4294971963 rc 0/0 00000100:00100000:2.0:1789760735.312857:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4673 00000100:00100000:2.0:1789760735.312866:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716515436328 time 716573704437 state 0x2 ref 0 00000100:00100000:0.0:1789760735.318139:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.318415:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224576 00000100:00100000:2.0:1789760735.318426:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224576 00000100:00100000:2.0:1789760735.318482:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716515436328 time 716573704437 state 0x0 ref 1 00000100:00100000:2.0:1789760735.318498:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716573964853 time 716573704437 state 0x1 ref 1 00000100:00100000:2.0:1789760735.318529:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716573964853 time 716633831712 state 0x0 ref 1 00000100:00100000:2.0:1789760735.318542:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.318588:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19021824, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.318598:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.318614:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.318619:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.318625:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.318639:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.345470:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971964 is committed 00002000:00010000:1.0:1789760735.345506:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19021824 -> 19025920 00010000:00010000:1.0:1789760735.345520:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345532:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345545:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345551:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760735.345556:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345657:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345681:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.345690:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7b37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.345847:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27291us (27702us total) trans 4294971964 rc 0/0 00000100:00100000:1.0:1789760735.345856:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4674 00000100:00100000:1.0:1789760735.345866:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716573964853 time 716633831712 state 0x2 ref 0 00000100:00100000:3.0:1789760735.348435:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760735.348736:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224704 00000100:00100000:1.0:1789760735.348745:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224704 00000100:00100000:1.0:1789760735.348793:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716573964853 time 716633831712 state 0x0 ref 1 00000100:00100000:1.0:1789760735.348807:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716634092128 time 716633831712 state 0x1 ref 1 00000100:00100000:1.0:1789760735.348849:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716634092128 time 716664151950 state 0x0 ref 1 00000100:00100000:1.0:1789760735.348861:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760735.348907:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19025920, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:1.0:1789760735.348915:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.348931:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760735.348936:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.348942:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.348957:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.386101:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971965 is committed 00002000:00010000:1.0:1789760735.386184:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19025920 -> 19030016 00010000:00010000:1.0:1789760735.386200:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386213:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386222:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386229:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760735.386234:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386243:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386264:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.386272:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7b3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.386471:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37607us (38038us total) trans 4294971965 rc 0/0 00000100:00100000:1.0:1789760735.386481:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4675 00000100:00100000:1.0:1789760735.386490:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716634092128 time 716664151950 state 0x2 ref 0 00000100:00100000:1.0:1789760735.390244:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.390397:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224832 00000100:00100000:2.0:1789760735.390407:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224832 00000100:00100000:2.0:1789760735.390464:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716634092128 time 716664151950 state 0x0 ref 1 00000100:00100000:2.0:1789760735.390479:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716664412366 time 716664151950 state 0x1 ref 1 00000100:00100000:2.0:1789760735.390513:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716664412366 time 716705817142 state 0x0 ref 1 00000100:00100000:2.0:1789760735.390526:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.390680:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19030016, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.390796:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.390815:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.390819:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.390826:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.390849:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.426204:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971966 is committed 00002000:00010000:0.0:1789760735.426315:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19030016 -> 19034112 00010000:00010000:0.0:1789760735.426330:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426344:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426354:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426361:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.426367:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426376:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426394:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.426403:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7b45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.426493:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507224832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35964us (36253us total) trans 4294971966 rc 0/0 00000100:00100000:0.0:1789760735.426503:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4676 00000100:00100000:0.0:1789760735.426513:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716664412366 time 716705817142 state 0x2 ref 0 00000100:00100000:0.0:1789760735.432865:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760735.433016:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507224960 00000100:00100000:1.0:1789760735.433026:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507224960 00000100:00100000:1.0:1789760735.433078:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716664412366 time 716705817142 state 0x0 ref 1 00000100:00100000:1.0:1789760735.433093:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716706077558 time 716705817142 state 0x1 ref 1 00000100:00100000:1.0:1789760735.433126:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716706077558 time 716748428591 state 0x0 ref 1 00000100:00100000:1.0:1789760735.433138:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760735.433186:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19034112, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:1.0:1789760735.433195:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.433209:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760735.433215:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.433221:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.433235:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.455161:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760735.455182:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271232:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760735.455239:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760735.455308:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271232 00000100:00100000:0.0:1789760735.455315:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271232 00000100:00100000:0.0:1789760735.455353:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789760735.455361:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538271232:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760735.455393:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760735.455428:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538271232:12345-0@lo:41:osp-pre-0-1.0 Request processed in 67us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760735.455435:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789760735.455472:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271232:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760735.481016:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971967 is committed 00002000:00010000:1.0:1789760735.481084:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19034112 -> 19038208 00010000:00010000:1.0:1789760735.481100:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481115:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481124:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481132:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760735.481136:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481348:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481370:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.481378:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7b4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.481475:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507224960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48333us (48613us total) trans 4294971967 rc 0/0 00000100:00100000:1.0:1789760735.481486:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4677 00000100:00100000:1.0:1789760735.481496:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716706077558 time 716748428591 state 0x2 ref 0 00000100:00100000:1.0:1789760735.485803:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760735.485980:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225088 00000100:00100000:0.0:1789760735.485990:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225088 00000100:00100000:0.0:1789760735.486042:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716706077558 time 716748428591 state 0x0 ref 1 00000100:00100000:0.0:1789760735.486056:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716748689007 time 716748428591 state 0x1 ref 1 00000100:00100000:0.0:1789760735.486092:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716748689007 time 716801394629 state 0x0 ref 1 00000100:00100000:0.0:1789760735.486104:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507225088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760735.486153:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19038208, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760735.486161:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.486177:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760735.486181:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.486187:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.486200:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.516675:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971968 is committed 00002000:00010000:0.0:1789760735.519145:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19038208 -> 19042304 00010000:00010000:0.0:1789760735.519167:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519186:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519196:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519202:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.519208:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519218:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519239:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.519246:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.519357:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507225088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33249us (33557us total) trans 4294971968 rc 0/0 00000100:00100000:0.0:1789760735.519367:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4678 00000100:00100000:0.0:1789760735.519376:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716748689007 time 716801394629 state 0x2 ref 0 00000100:00100000:3.0:1789760735.523179:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.523357:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225216 00000100:00100000:2.0:1789760735.523367:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225216 00000100:00100000:2.0:1789760735.523422:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716748689007 time 716801394629 state 0x0 ref 1 00000100:00100000:2.0:1789760735.523437:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716801655045 time 716801394629 state 0x1 ref 1 00000100:00100000:2.0:1789760735.523465:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716801655045 time 716838768202 state 0x0 ref 1 00000100:00100000:2.0:1789760735.523478:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.523526:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19042304, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.523535:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.523551:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.523555:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.523562:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.523575:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.557669:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971969 is committed 00002000:00010000:2.0:1789760735.557857:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19042304 -> 19046400 00010000:00010000:2.0:1789760735.557935:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.557949:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.557957:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.557964:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.557968:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.557978:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.558001:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.558008:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.558136:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34655us (34961us total) trans 4294971969 rc 0/0 00000100:00100000:2.0:1789760735.558145:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4679 00000100:00100000:2.0:1789760735.558154:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716801655045 time 716838768202 state 0x2 ref 0 00000100:00100000:3.0:1789760735.568129:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760735.568235:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225344 00000100:00100000:1.0:1789760735.568245:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225344 00000100:00100000:1.0:1789760735.568599:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716801655045 time 716838768202 state 0x0 ref 1 00000100:00100000:1.0:1789760735.568614:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716839028618 time 716838768202 state 0x1 ref 1 00000100:00100000:1.0:1789760735.568679:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716839028618 time 716883982128 state 0x0 ref 1 00000100:00100000:1.0:1789760735.568734:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760735.568780:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19046400, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:1.0:1789760735.568788:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.568805:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760735.568810:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.568816:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.568829:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.599191:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971970 is committed 00002000:00010000:3.0:1789760735.599304:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19046400 -> 19050496 00010000:00010000:3.0:1789760735.599319:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599332:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599342:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599350:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760735.599354:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599364:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599383:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.599392:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7b61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.599479:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30784us (31355us total) trans 4294971970 rc 0/0 00000100:00100000:3.0:1789760735.599492:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4680 00000100:00100000:3.0:1789760735.599502:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716839028618 time 716883982128 state 0x2 ref 0 00000100:00100000:1.0:1789760735.603380:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.603509:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225472 00000100:00100000:2.0:1789760735.603518:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225472 00000100:00100000:2.0:1789760735.603572:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716839028618 time 716883982128 state 0x0 ref 1 00000100:00100000:2.0:1789760735.603587:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716884242544 time 716883982128 state 0x1 ref 1 00000100:00100000:2.0:1789760735.603615:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716884242544 time 716918918752 state 0x0 ref 1 00000100:00100000:2.0:1789760735.603627:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.603673:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19050496, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.603681:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.603734:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.603741:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.603747:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.603761:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.633050:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971971 is committed 00002000:00010000:2.0:1789760735.633146:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19050496 -> 19054592 00010000:00010000:2.0:1789760735.633161:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633174:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633183:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633189:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.633194:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633203:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633221:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.633235:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.633326:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29696us (29951us total) trans 4294971971 rc 0/0 00000100:00100000:2.0:1789760735.633336:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4681 00000100:00100000:2.0:1789760735.633344:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716884242544 time 716918918752 state 0x2 ref 0 00000100:00100000:1.0:1789760735.636953:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.637020:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225600 00000100:00100000:2.0:1789760735.637029:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225600 00000100:00100000:2.0:1789760735.637081:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716884242544 time 716918918752 state 0x0 ref 1 00000100:00100000:2.0:1789760735.637097:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716919179168 time 716918918752 state 0x1 ref 1 00000100:00100000:2.0:1789760735.637125:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716919179168 time 716952428879 state 0x0 ref 1 00000100:00100000:2.0:1789760735.637137:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.637179:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19054592, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.637187:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.637204:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760735.637208:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.637214:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.637234:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.671622:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971972 is committed 00002000:00010000:2.0:1789760735.671782:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19054592 -> 19058688 00010000:00010000:2.0:1789760735.671797:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671811:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671820:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671828:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760735.671842:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671852:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671877:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.671885:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.671972:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34831us (35024us total) trans 4294971972 rc 0/0 00000100:00100000:2.0:1789760735.671982:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4682 00000100:00100000:2.0:1789760735.671991:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716919179168 time 716952428879 state 0x2 ref 0 00000100:00100000:3.0:1789760735.675506:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760735.675585:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225728 00000100:00100000:1.0:1789760735.675595:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225728 00000100:00100000:1.0:1789760735.675648:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716919179168 time 716952428879 state 0x0 ref 1 00000100:00100000:1.0:1789760735.675663:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716952689295 time 716952428879 state 0x1 ref 1 00000100:00100000:1.0:1789760735.675720:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716952689295 time 716990994868 state 0x0 ref 1 00000100:00100000:1.0:1789760735.675734:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760735.675780:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19058688, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:1.0:1789760735.675789:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.675810:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760735.675815:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.675822:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.675846:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.733880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971973 is committed 00002000:00010000:3.0:1789760735.733991:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19058688 -> 19062784 00010000:00010000:3.0:1789760735.734006:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734018:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734028:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734035:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760735.734040:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734049:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734069:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.734077:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7b76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.734167:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507225728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58432us (58667us total) trans 4294971973 rc 0/0 00000100:00100000:3.0:1789760735.734178:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4683 00000100:00100000:3.0:1789760735.734187:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716952689295 time 716990994868 state 0x2 ref 0 00000100:00100000:1.0:1789760735.749647:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.749803:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225856 00000100:00100000:2.0:1789760735.749817:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225856 00000100:00100000:2.0:1789760735.749892:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716952689295 time 716990994868 state 0x0 ref 1 00000100:00100000:2.0:1789760735.749907:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716991255284 time 716990994868 state 0x1 ref 1 00000100:00100000:2.0:1789760735.749938:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716991255284 time 717065241049 state 0x0 ref 1 00000100:00100000:2.0:1789760735.749951:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507225856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.750024:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19062784, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.750033:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.750048:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760735.750053:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.750059:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.750073:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760735.803613:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19062784 -> 19066880 00010000:00010000:0.0:1789760735.803632:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803643:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803652:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803664:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.803669:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803677:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803722:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.803729:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b7b7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.803816:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507225856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53863us (54183us total) trans 4294971974 rc 0/0 00000100:00100000:0.0:1789760735.803826:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4684 00000100:00100000:0.0:1789760735.803845:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716991255284 time 717065241049 state 0x2 ref 0 00000001:00080000:3.0:1789760735.805681:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971974 is committed 00000100:00100000:1.0:1789760735.807774:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760735.807901:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507225984 00000100:00100000:2.0:1789760735.807910:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507225984 00000100:00100000:2.0:1789760735.807958:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 716991255284 time 717065241049 state 0x0 ref 1 00000100:00100000:2.0:1789760735.807972:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717065501465 time 717065241049 state 0x1 ref 1 00000100:00100000:2.0:1789760735.807999:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717065501465 time 717123302833 state 0x0 ref 1 00000100:00100000:2.0:1789760735.808010:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507225984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760735.808057:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19066880, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:2.0:1789760735.808065:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.808080:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760735.808084:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.808090:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.808104:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.842061:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971975 is committed 00002000:00010000:3.0:1789760735.842163:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19066880 -> 19070976 00010000:00010000:3.0:1789760735.842179:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842191:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842200:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842207:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760735.842212:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842221:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842242:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.842256:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7b84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.842405:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507225984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34392us (34635us total) trans 4294971975 rc 0/0 00000100:00100000:3.0:1789760735.842415:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4685 00000100:00100000:3.0:1789760735.842424:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717065501465 time 717123302833 state 0x2 ref 0 00000100:00100000:1.0:1789760735.846951:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760735.847335:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226112 00000100:00100000:0.0:1789760735.847345:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226112 00000100:00100000:0.0:1789760735.847396:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717065501465 time 717123302833 state 0x0 ref 1 00000100:00100000:0.0:1789760735.847411:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717123563249 time 717123302833 state 0x1 ref 1 00000100:00100000:0.0:1789760735.847470:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717123563249 time 717162772454 state 0x0 ref 1 00000100:00100000:0.0:1789760735.847483:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760735.847530:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19070976, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760735.847538:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.847554:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760735.847559:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.847565:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.847579:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.881946:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971976 is committed 00002000:00010000:1.0:1789760735.882180:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19070976 -> 19075072 00010000:00010000:1.0:1789760735.882195:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882207:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882216:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882222:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760735.882227:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882242:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882263:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.882271:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7b8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.882377:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34883us (35676us total) trans 4294971976 rc 0/0 00000100:00100000:1.0:1789760735.882388:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4686 00000100:00100000:1.0:1789760735.882397:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717123563249 time 717162772454 state 0x2 ref 0 00000100:00100000:1.0:1789760735.885893:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760735.886005:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226240 00000100:00100000:0.0:1789760735.886015:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226240 00000100:00100000:0.0:1789760735.886065:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717123563249 time 717162772454 state 0x0 ref 1 00000100:00100000:0.0:1789760735.886081:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717163032870 time 717162772454 state 0x1 ref 1 00000100:00100000:0.0:1789760735.886110:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717163032870 time 717201412858 state 0x0 ref 1 00000100:00100000:0.0:1789760735.886122:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760735.886170:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19075072, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760735.886178:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.886199:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760735.886204:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.886210:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.886223:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.917808:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19075072 -> 19079168 00010000:00010000:1.0:1789760735.917827:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917850:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917859:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917879:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760735.917885:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917894:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917912:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.917919:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7b92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.918004:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507226240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31879us (32116us total) trans 4294971977 rc 0/0 00000100:00100000:1.0:1789760735.918014:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4687 00000100:00100000:1.0:1789760735.918023:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717163032870 time 717201412858 state 0x2 ref 0 00000001:00080000:3.0:1789760735.920553:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971977 is committed 00000100:00100000:0.0:1789760735.921610:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760735.921689:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226368 00000100:00100000:3.0:1789760735.921735:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226368 00000100:00100000:3.0:1789760735.921789:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717163032870 time 717201412858 state 0x0 ref 1 00000100:00100000:3.0:1789760735.921804:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717201673274 time 717201412858 state 0x1 ref 1 00000100:00100000:3.0:1789760735.921828:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717201673274 time 717237131531 state 0x0 ref 1 00000100:00100000:3.0:1789760735.921850:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760735.921906:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19079168, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:3.0:1789760735.921915:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.921930:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760735.921935:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.921942:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.921957:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.943019:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971978 is committed 00002000:00010000:0.0:1789760735.943120:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19079168 -> 19083264 00010000:00010000:0.0:1789760735.943135:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943150:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943159:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943166:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.943171:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943180:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943204:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.943211:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7b99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.943308:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21455us (21698us total) trans 4294971978 rc 0/0 00000100:00100000:0.0:1789760735.943318:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4688 00000100:00100000:0.0:1789760735.943329:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717201673274 time 717237131531 state 0x2 ref 0 00000100:00100000:0.0:1789760735.947475:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760735.947560:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226496 00000100:00100000:1.0:1789760735.947570:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226496 00000100:00100000:1.0:1789760735.947619:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717201673274 time 717237131531 state 0x0 ref 1 00000100:00100000:1.0:1789760735.947634:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717237391947 time 717237131531 state 0x1 ref 1 00000100:00100000:1.0:1789760735.947657:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717237391947 time 717262960674 state 0x0 ref 1 00000100:00100000:1.0:1789760735.947670:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760735.947746:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19083264, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:1.0:1789760735.947754:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.947769:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760735.947773:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.947780:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.947793:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.978675:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971979 is committed 00002000:00010000:0.0:1789760735.978846:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19083264 -> 19087360 00010000:00010000:0.0:1789760735.978862:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.978945:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.978955:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.978964:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760735.978969:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.978983:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.979004:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.979012:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7ba0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.979146:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31472us (31674us total) trans 4294971979 rc 0/0 00000100:00100000:0.0:1789760735.979158:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4689 00000100:00100000:0.0:1789760735.979168:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717237391947 time 717262960674 state 0x2 ref 0 00000100:00100000:1.0:1789760735.982546:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760735.982651:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226624 00000100:00100000:0.0:1789760735.982661:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226624 00000100:00100000:0.0:1789760735.982777:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717237391947 time 717262960674 state 0x0 ref 1 00000100:00100000:0.0:1789760735.982792:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717263221090 time 717262960674 state 0x1 ref 1 00000100:00100000:0.0:1789760735.982863:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717263221090 time 717298165653 state 0x0 ref 1 00000100:00100000:0.0:1789760735.982933:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760735.982981:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19087360, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760735.982991:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.983007:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760735.983012:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.983018:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.983033:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.012265:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971980 is committed 00002000:00010000:0.0:1789760736.012411:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19087360 -> 19091456 00010000:00010000:0.0:1789760736.012426:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012439:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012448:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012455:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.012459:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012468:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012490:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.012497:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7ba7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.012738:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29804us (30195us total) trans 4294971980 rc 0/0 00000100:00100000:0.0:1789760736.012749:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4690 00000100:00100000:0.0:1789760736.012758:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717263221090 time 717298165653 state 0x2 ref 0 00000100:00100000:1.0:1789760736.017401:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.017475:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226752 00000100:00100000:0.0:1789760736.017485:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226752 00000100:00100000:0.0:1789760736.017537:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717263221090 time 717298165653 state 0x0 ref 1 00000100:00100000:0.0:1789760736.017554:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717298426069 time 717298165653 state 0x1 ref 1 00000100:00100000:0.0:1789760736.017584:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717298426069 time 717332886497 state 0x0 ref 1 00000100:00100000:0.0:1789760736.017597:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.017643:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19091456, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0x9807 00010000:00010000:0.0:1789760736.017652:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.017669:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.017674:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.017680:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.017738:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.046516:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971981 is committed 00002000:00010000:3.0:1789760736.046608:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19091456 -> 19095552 00002000:00010000:3.0:1789760736.046612:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:3.0:1789760736.046615:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:3.0:1789760736.046628:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046641:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046651:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046658:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.046664:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046673:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046707:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.046716:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7bae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.046847:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507226752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29248us (29451us total) trans 4294971981 rc 0/0 00000100:00100000:3.0:1789760736.046858:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4691 00000100:00100000:3.0:1789760736.046883:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717298426069 time 717332886497 state 0x2 ref 0 00000100:00100000:1.0:1789760736.055416:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.055503:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507226880 00000100:00100000:0.0:1789760736.055514:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507226880 00000100:00100000:0.0:1789760736.055566:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717298426069 time 717332886497 state 0x0 ref 1 00000100:00100000:0.0:1789760736.055582:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717333146913 time 717332886497 state 0x1 ref 1 00000100:00100000:0.0:1789760736.055605:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717333146913 time 717370908744 state 0x0 ref 1 00000100:00100000:0.0:1789760736.055618:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.055666:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19095552, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.055675:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.055722:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.055728:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.055735:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.055749:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.114060:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971982 is committed 00002000:00010000:3.0:1789760736.114164:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19095552 -> 19099648 00010000:00010000:3.0:1789760736.114178:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114193:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114203:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114210:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.114215:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114225:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114250:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.114258:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b7bb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.114360:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507226880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58739us (58949us total) trans 4294971982 rc 0/0 00000100:00100000:3.0:1789760736.114370:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4692 00000100:00100000:3.0:1789760736.114379:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717333146913 time 717370908744 state 0x2 ref 0 00000100:00100000:1.0:1789760736.119975:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.120105:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227008 00000100:00100000:0.0:1789760736.120115:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227008 00000100:00100000:0.0:1789760736.120177:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717333146913 time 717370908744 state 0x0 ref 1 00000100:00100000:0.0:1789760736.120193:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717371169160 time 717370908744 state 0x1 ref 1 00000100:00100000:0.0:1789760736.120220:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717371169160 time 717435524123 state 0x0 ref 1 00000100:00100000:0.0:1789760736.120232:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507227008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.120294:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19099648, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.120302:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.120319:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.120323:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.120330:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.120343:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.159778:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760736.191934:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19099648 -> 19103744 00010000:00010000:2.0:1789760736.191953:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.191970:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.191979:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.191986:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760736.191991:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.192000:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.192019:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.192027:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b7bbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.192130:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507227008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71896us (72161us total) trans 4294971983 rc 0/0 00000100:00100000:2.0:1789760736.192141:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4693 00000100:00100000:2.0:1789760736.192151:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717371169160 time 717435524123 state 0x2 ref 0 00000001:00080000:3.0:1789760736.192230:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971983 is committed 00000100:00100000:0.0:1789760736.199762:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760736.199856:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227136 00000100:00100000:3.0:1789760736.199874:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227136 00000100:00100000:3.0:1789760736.199934:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717371169160 time 717435524123 state 0x0 ref 1 00000100:00100000:3.0:1789760736.199948:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717435784539 time 717435524123 state 0x1 ref 1 00000100:00100000:3.0:1789760736.199980:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717435784539 time 717515283786 state 0x0 ref 1 00000100:00100000:3.0:1789760736.199996:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760736.200052:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19103744, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:3.0:1789760736.200061:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.200088:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760736.200093:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.200102:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.200118:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.239022:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19103744 -> 19107840 00010000:00010000:3.0:1789760736.239042:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239056:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239066:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239073:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.239078:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239088:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239107:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.239116:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.239207:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507227136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39208us (39452us total) trans 4294971984 rc 0/0 00000100:00100000:3.0:1789760736.239217:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4694 00000100:00100000:3.0:1789760736.239227:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717435784539 time 717515283786 state 0x2 ref 0 00000001:00080000:0.0:1789760736.239296:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971984 is committed 00000100:00100000:1.0:1789760736.242785:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760736.242915:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227264 00000100:00100000:3.0:1789760736.242925:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227264 00000100:00100000:3.0:1789760736.242976:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717435784539 time 717515283786 state 0x0 ref 1 00000100:00100000:3.0:1789760736.242993:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717515544202 time 717515283786 state 0x1 ref 1 00000100:00100000:3.0:1789760736.243025:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717515544202 time 717558328797 state 0x0 ref 1 00000100:00100000:3.0:1789760736.243038:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507227264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760736.243084:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19107840, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:3.0:1789760736.243092:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.243109:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760736.243113:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.243120:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.243139:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.271496:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971985 is committed 00002000:00010000:3.0:1789760736.271591:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19107840 -> 19111936 00010000:00010000:3.0:1789760736.271606:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271619:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271629:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271636:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.271641:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271650:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271668:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.271676:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.271787:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507227264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28747us (29009us total) trans 4294971985 rc 0/0 00000100:00100000:3.0:1789760736.271798:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4695 00000100:00100000:3.0:1789760736.271807:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717515544202 time 717558328797 state 0x2 ref 0 00000100:00100000:2.0:1789760736.275606:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.275737:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227392 00000100:00100000:0.0:1789760736.275746:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227392 00000100:00100000:0.0:1789760736.275801:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717515544202 time 717558328797 state 0x0 ref 1 00000100:00100000:0.0:1789760736.275818:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717558589213 time 717558328797 state 0x1 ref 1 00000100:00100000:0.0:1789760736.275851:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717558589213 time 717591154402 state 0x0 ref 1 00000100:00100000:0.0:1789760736.275873:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.275922:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19111936, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.275930:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.275946:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.275950:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.275957:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.275970:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.305907:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971986 is committed 00002000:00010000:0.0:1789760736.306119:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19111936 -> 19116032 00010000:00010000:0.0:1789760736.306133:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306145:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306154:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306168:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.306173:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306181:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306199:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.306207:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7bd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.306297:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30431us (30696us total) trans 4294971986 rc 0/0 00000100:00100000:0.0:1789760736.306308:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4696 00000100:00100000:0.0:1789760736.306318:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717558589213 time 717591154402 state 0x2 ref 0 00000100:00100000:0.0:1789760736.310082:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760736.310164:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227520 00000100:00100000:3.0:1789760736.310173:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227520 00000100:00100000:3.0:1789760736.310224:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717558589213 time 717591154402 state 0x0 ref 1 00000100:00100000:3.0:1789760736.310239:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717591414818 time 717591154402 state 0x1 ref 1 00000100:00100000:3.0:1789760736.310300:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717591414818 time 717625603132 state 0x0 ref 1 00000100:00100000:3.0:1789760736.310313:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760736.310359:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19116032, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:3.0:1789760736.310368:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.310384:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760736.310389:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.310395:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.310409:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.338762:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971987 is committed 00002000:00010000:3.0:1789760736.338885:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19116032 -> 19120128 00010000:00010000:3.0:1789760736.338901:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338916:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338925:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338932:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.338937:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338945:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338965:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.338974:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7bd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.339065:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28749us (28986us total) trans 4294971987 rc 0/0 00000100:00100000:3.0:1789760736.339078:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4697 00000100:00100000:3.0:1789760736.339088:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717591414818 time 717625603132 state 0x2 ref 0 00000100:00100000:2.0:1789760736.343389:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760736.343466:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227648 00000100:00100000:3.0:1789760736.343475:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227648 00000100:00100000:3.0:1789760736.343524:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717591414818 time 717625603132 state 0x0 ref 1 00000100:00100000:3.0:1789760736.343538:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717625863548 time 717625603132 state 0x1 ref 1 00000100:00100000:3.0:1789760736.343570:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717625863548 time 717658872971 state 0x0 ref 1 00000100:00100000:3.0:1789760736.343582:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760736.343626:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19120128, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:3.0:1789760736.343635:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.343651:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760736.343656:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.343662:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.343675:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.382477:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971988 is committed 00002000:00010000:3.0:1789760736.382594:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19120128 -> 19124224 00010000:00010000:3.0:1789760736.382610:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382624:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382634:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382642:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760736.382646:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382656:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382677:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.382685:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.382990:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39404us (39603us total) trans 4294971988 rc 0/0 00000100:00100000:3.0:1789760736.383000:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4698 00000100:00100000:3.0:1789760736.383009:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717625863548 time 717658872971 state 0x2 ref 0 00000100:00100000:2.0:1789760736.387408:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.387512:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227776 00000100:00100000:0.0:1789760736.387521:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227776 00000100:00100000:0.0:1789760736.387575:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717625863548 time 717658872971 state 0x0 ref 1 00000100:00100000:0.0:1789760736.387590:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717659133387 time 717658872971 state 0x1 ref 1 00000100:00100000:0.0:1789760736.387658:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717659133387 time 717702960132 state 0x0 ref 1 00000100:00100000:0.0:1789760736.387672:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507227776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.387816:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19124224, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.387834:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.387851:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.387855:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.387924:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.387941:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.422090:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19124224 -> 19128320 00010000:00010000:0.0:1789760736.422110:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422124:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422133:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422141:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.422146:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422155:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422175:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.422182:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7be6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.422426:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507227776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34751us (35022us total) trans 4294971989 rc 0/0 00000100:00100000:0.0:1789760736.422437:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4699 00000100:00100000:0.0:1789760736.422447:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717659133387 time 717702960132 state 0x2 ref 0 00000100:00100000:1.0:1789760736.426495:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760736.426533:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507227904 00000100:00100000:1.0:1789760736.426541:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507227904 00000100:00100000:1.0:1789760736.426593:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717659133387 time 717702960132 state 0x0 ref 1 00000100:00100000:1.0:1789760736.426607:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717703220548 time 717702960132 state 0x1 ref 1 00000100:00100000:1.0:1789760736.426627:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717703220548 time 717741931600 state 0x0 ref 1 00000100:00100000:1.0:1789760736.426640:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507227904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760736.426685:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19128320, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:1.0:1789760736.426732:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.426748:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760736.426753:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.426758:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.426773:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.427903:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971989 is committed 00002000:00010000:0.0:1789760736.464806:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19128320 -> 19132416 00010000:00010000:0.0:1789760736.464836:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464850:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464869:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464883:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.464888:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464898:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464917:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.464925:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7bed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.465014:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507227904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38372us (38523us total) trans 4294971990 rc 0/0 00000100:00100000:0.0:1789760736.465024:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4700 00000100:00100000:0.0:1789760736.465033:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717703220548 time 717741931600 state 0x2 ref 0 00000001:00080000:2.0:1789760736.466106:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971990 is committed 00000100:00100000:1.0:1789760736.468973:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760736.469066:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228032 00000100:00100000:3.0:1789760736.469075:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228032 00000100:00100000:3.0:1789760736.469128:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717703220548 time 717741931600 state 0x0 ref 1 00000100:00100000:3.0:1789760736.469144:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717742192016 time 717741931600 state 0x1 ref 1 00000100:00100000:3.0:1789760736.469206:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717742192016 time 717784508881 state 0x0 ref 1 00000100:00100000:3.0:1789760736.469219:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507228032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760736.469264:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19132416, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:3.0:1789760736.469336:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.469353:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760736.469357:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.469363:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.469377:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.497574:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971991 is committed 00002000:00010000:0.0:1789760736.497671:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19132416 -> 19136512 00010000:00010000:0.0:1789760736.497686:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497727:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497742:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497749:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.497754:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497763:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497784:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.497793:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7bf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.497954:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507228032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28732us (28986us total) trans 4294971991 rc 0/0 00000100:00100000:0.0:1789760736.497964:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4701 00000100:00100000:0.0:1789760736.497973:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717742192016 time 717784508881 state 0x2 ref 0 00000100:00100000:2.0:1789760736.501994:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.502086:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228160 00000100:00100000:0.0:1789760736.502095:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228160 00000100:00100000:0.0:1789760736.502148:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717742192016 time 717784508881 state 0x0 ref 1 00000100:00100000:0.0:1789760736.502164:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717784769297 time 717784508881 state 0x1 ref 1 00000100:00100000:0.0:1789760736.502234:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717784769297 time 717817536343 state 0x0 ref 1 00000100:00100000:0.0:1789760736.502247:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507228160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.502304:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19136512, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.502314:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.502328:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.502332:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.502339:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.502352:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.536092:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19136512 -> 19140608 00010000:00010000:0.0:1789760736.536111:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536125:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536135:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536143:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.536148:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536158:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536178:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.536186:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7bfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.536268:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507228160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34018us (34279us total) trans 4294971992 rc 0/0 00000001:00080000:2.0:1789760736.536467:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971992 is committed 00000100:00100000:0.0:1789760736.536521:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4702 00000100:00100000:0.0:1789760736.536530:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717784769297 time 717817536343 state 0x2 ref 0 00000100:00100000:2.0:1789760736.539356:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.539633:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228288 00000100:00100000:0.0:1789760736.539643:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228288 00000100:00100000:0.0:1789760736.539812:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717784769297 time 717817536343 state 0x0 ref 1 00000100:00100000:0.0:1789760736.539840:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717817796759 time 717817536343 state 0x1 ref 1 00000100:00100000:0.0:1789760736.539919:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717817796759 time 717855221941 state 0x0 ref 1 00000100:00100000:0.0:1789760736.539932:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.539976:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19140608, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.539986:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.540001:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.540005:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.540012:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.540026:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.566078:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971993 is committed 00002000:00010000:0.0:1789760736.566147:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19140608 -> 19144704 00010000:00010000:0.0:1789760736.566162:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566176:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566185:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566192:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.566197:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566206:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566225:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.566232:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19140608->19144703] (req 19140608->19144703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.566538:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26604us (27187us total) trans 4294971993 rc 0/0 00000100:00100000:0.0:1789760736.566549:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4703 00000100:00100000:0.0:1789760736.566559:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717817796759 time 717855221941 state 0x2 ref 0 00000100:00100000:1.0:1789760736.571157:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.571242:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228416 00000100:00100000:0.0:1789760736.571252:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228416 00000100:00100000:0.0:1789760736.571364:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717817796759 time 717855221941 state 0x0 ref 1 00000100:00100000:0.0:1789760736.571379:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717855482357 time 717855221941 state 0x1 ref 1 00000100:00100000:0.0:1789760736.571451:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717855482357 time 717886753259 state 0x0 ref 1 00000100:00100000:0.0:1789760736.571465:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.571514:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19144704, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.571523:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.571539:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.571544:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.571550:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.571563:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.597607:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971994 is committed 00002000:00010000:1.0:1789760736.597987:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19144704 -> 19148800 00010000:00010000:1.0:1789760736.598003:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598026:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598035:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598042:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760736.598049:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598221:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598246:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.598255:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7c09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.598360:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26893us (27207us total) trans 4294971994 rc 0/0 00000100:00100000:1.0:1789760736.598371:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4704 00000100:00100000:1.0:1789760736.598382:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717855482357 time 717886753259 state 0x2 ref 0 00000100:00100000:2.0:1789760736.602238:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.602859:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228544 00000100:00100000:0.0:1789760736.602871:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228544 00000100:00100000:0.0:1789760736.602930:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717855482357 time 717886753259 state 0x0 ref 1 00000100:00100000:0.0:1789760736.602946:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717887013675 time 717886753259 state 0x1 ref 1 00000100:00100000:0.0:1789760736.602980:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717887013675 time 717918283427 state 0x0 ref 1 00000100:00100000:0.0:1789760736.602993:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507228544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.603055:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19148800, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.603065:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.603081:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.603086:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.603093:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.603108:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.638292:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19148800 -> 19152896 00010000:00010000:0.0:1789760736.638313:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638325:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638335:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638343:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.638349:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638359:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638380:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.638388:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.638520:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507228544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35526us (36289us total) trans 4294971995 rc 0/0 00000100:00100000:0.0:1789760736.638534:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4705 00000100:00100000:0.0:1789760736.638543:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717887013675 time 717918283427 state 0x2 ref 0 00000001:00080000:0.0:1789760736.638583:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971995 is committed 00000100:00100000:2.0:1789760736.642637:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.643015:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228672 00000100:00100000:0.0:1789760736.643024:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228672 00000100:00100000:0.0:1789760736.643077:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717887013675 time 717918283427 state 0x0 ref 1 00000100:00100000:0.0:1789760736.643092:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717918543843 time 717918283427 state 0x1 ref 1 00000100:00100000:0.0:1789760736.643154:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717918543843 time 717958456929 state 0x0 ref 1 00000100:00100000:0.0:1789760736.643168:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507228672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.643212:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19152896, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.643221:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.643235:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760736.643239:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.643246:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.643260:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.678766:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971996 is committed 00002000:00010000:0.0:1789760736.678970:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19152896 -> 19156992 00010000:00010000:0.0:1789760736.678986:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679000:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679009:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679016:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.679021:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679031:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679050:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.679058:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7c17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.679184:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507228672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36013us (36552us total) trans 4294971996 rc 0/0 00000100:00100000:0.0:1789760736.679194:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4706 00000100:00100000:0.0:1789760736.679204:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717918543843 time 717958456929 state 0x2 ref 0 00000100:00100000:2.0:1789760736.685443:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760736.685525:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228800 00000100:00100000:1.0:1789760736.685534:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228800 00000100:00100000:1.0:1789760736.685587:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717918543843 time 717958456929 state 0x0 ref 1 00000100:00100000:1.0:1789760736.685602:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717958717345 time 717958456929 state 0x1 ref 1 00000100:00100000:1.0:1789760736.685634:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717958717345 time 718000937568 state 0x0 ref 1 00000100:00100000:1.0:1789760736.685652:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507228800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760736.685738:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19156992, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:1.0:1789760736.685748:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.685763:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760736.685768:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.685774:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.685790:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.737277:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971997 is committed 00002000:00010000:1.0:1789760736.737566:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19156992 -> 19161088 00010000:00010000:1.0:1789760736.737582:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737596:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737605:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737612:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760736.737617:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737626:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737646:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.737654:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.737835:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507228800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52168us (52383us total) trans 4294971997 rc 0/0 00000100:00100000:1.0:1789760736.737846:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4707 00000100:00100000:1.0:1789760736.737856:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717958717345 time 718000937568 state 0x2 ref 0 00000100:00100000:2.0:1789760736.741267:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.741508:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507228928 00000100:00100000:0.0:1789760736.741517:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507228928 00000100:00100000:0.0:1789760736.741571:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 717958717345 time 718000937568 state 0x0 ref 1 00000100:00100000:0.0:1789760736.741586:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718001197984 time 718000937568 state 0x1 ref 1 00000100:00100000:0.0:1789760736.741653:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718001197984 time 718056955952 state 0x0 ref 1 00000100:00100000:0.0:1789760736.741666:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.741750:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19161088, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.741759:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.741774:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760736.741778:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.741783:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.741797:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.773815:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971998 is committed 00002000:00010000:0.0:1789760736.773933:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19161088 -> 19165184 00010000:00010000:0.0:1789760736.773948:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.773967:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.773977:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.773984:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.773990:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.773999:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.774017:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.774025:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7c25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.774113:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507228928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32444us (32851us total) trans 4294971998 rc 0/0 00000100:00100000:0.0:1789760736.774123:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4708 00000100:00100000:0.0:1789760736.774132:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718001197984 time 718056955952 state 0x2 ref 0 00000100:00100000:0.0:1789760736.777254:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760736.777436:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229056 00000100:00100000:2.0:1789760736.777448:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229056 00000100:00100000:2.0:1789760736.777501:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718001197984 time 718056955952 state 0x0 ref 1 00000100:00100000:2.0:1789760736.777517:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718057216368 time 718056955952 state 0x1 ref 1 00000100:00100000:2.0:1789760736.777548:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718057216368 time 718092850594 state 0x0 ref 1 00000100:00100000:2.0:1789760736.777561:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507229056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760736.777607:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19165184, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:2.0:1789760736.777616:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.777632:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760736.777637:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.777645:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.777659:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.845748:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971999 is committed 00002000:00010000:1.0:1789760736.845871:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19165184 -> 19169280 00010000:00010000:1.0:1789760736.845886:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845898:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845906:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845914:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760736.845919:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845934:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845952:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.845961:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7c2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.846052:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507229056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68488us (68802us total) trans 4294971999 rc 0/0 00000100:00100000:1.0:1789760736.846063:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4709 00000100:00100000:1.0:1789760736.846072:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718057216368 time 718092850594 state 0x2 ref 0 00000100:00100000:2.0:1789760736.852242:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760736.852343:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229184 00000100:00100000:1.0:1789760736.852354:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229184 00000100:00100000:1.0:1789760736.852407:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718057216368 time 718092850594 state 0x0 ref 1 00000100:00100000:1.0:1789760736.852422:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718093111010 time 718092850594 state 0x1 ref 1 00000100:00100000:1.0:1789760736.852486:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718093111010 time 718167789141 state 0x0 ref 1 00000100:00100000:1.0:1789760736.852498:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760736.852544:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19169280, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:1.0:1789760736.852553:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.852568:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760736.852572:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.852578:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.852591:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.882453:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972000 is committed 00002000:00010000:1.0:1789760736.882535:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19169280 -> 19173376 00010000:00010000:1.0:1789760736.882552:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882564:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882573:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882580:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760736.882585:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882594:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882614:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.882623:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.882780:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30277us (30542us total) trans 4294972000 rc 0/0 00000100:00100000:1.0:1789760736.882790:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4710 00000100:00100000:1.0:1789760736.882800:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718093111010 time 718167789141 state 0x2 ref 0 00000100:00100000:2.0:1789760736.890792:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760736.891116:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229312 00000100:00100000:0.0:1789760736.891126:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229312 00000100:00100000:0.0:1789760736.891182:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718093111010 time 718167789141 state 0x0 ref 1 00000100:00100000:0.0:1789760736.891198:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718168049557 time 718167789141 state 0x1 ref 1 00000100:00100000:0.0:1789760736.891230:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718168049557 time 718206532935 state 0x0 ref 1 00000100:00100000:0.0:1789760736.891243:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760736.891291:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19173376, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760736.891301:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.891316:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760736.891321:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.891327:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.891341:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.940565:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19173376 -> 19177472 00010000:00010000:0.0:1789760736.940585:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940598:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940609:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940617:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760736.940622:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940632:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940650:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.940658:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.940779:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507229312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49533us (49994us total) trans 4294972001 rc 0/0 00000100:00100000:0.0:1789760736.940790:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4711 00000100:00100000:0.0:1789760736.940799:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718168049557 time 718206532935 state 0x2 ref 0 00000001:00080000:1.0:1789760736.941035:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972001 is committed 00000100:00100000:0.0:1789760736.943770:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760736.944045:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229440 00000100:00100000:1.0:1789760736.944054:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229440 00000100:00100000:1.0:1789760736.944108:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718168049557 time 718206532935 state 0x0 ref 1 00000100:00100000:1.0:1789760736.944122:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718206793351 time 718206532935 state 0x1 ref 1 00000100:00100000:1.0:1789760736.944150:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718206793351 time 718259453528 state 0x0 ref 1 00000100:00100000:1.0:1789760736.944161:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760736.944205:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19177472, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:1.0:1789760736.944213:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.944229:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760736.944233:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.944239:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.944253:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.977608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972002 is committed 00002000:00010000:1.0:1789760736.977715:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19177472 -> 19181568 00010000:00010000:1.0:1789760736.977729:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977743:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977752:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977759:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760736.977764:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977773:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977792:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.977799:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7c41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.977906:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33742us (34142us total) trans 4294972002 rc 0/0 00000100:00100000:1.0:1789760736.977916:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4712 00000100:00100000:1.0:1789760736.977926:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718206793351 time 718259453528 state 0x2 ref 0 00000100:00100000:3.0:1789760736.981005:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760736.981426:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229568 00000100:00100000:1.0:1789760736.981436:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229568 00000100:00100000:1.0:1789760736.981487:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718206793351 time 718259453528 state 0x0 ref 1 00000100:00100000:1.0:1789760736.981502:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718259713944 time 718259453528 state 0x1 ref 1 00000100:00100000:1.0:1789760736.981529:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718259713944 time 718296832345 state 0x0 ref 1 00000100:00100000:1.0:1789760736.981541:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507229568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760736.981586:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19181568, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:1.0:1789760736.981595:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.981610:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760736.981614:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.981620:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.981634:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.021181:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972003 is committed 00002000:00010000:0.0:1789760737.021259:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19181568 -> 19185664 00010000:00010000:0.0:1789760737.021329:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021348:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021357:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021363:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760737.021368:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021377:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021396:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.021403:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b7c48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.021532:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507229568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39988us (40535us total) trans 4294972003 rc 0/0 00000100:00100000:0.0:1789760737.021543:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4713 00000100:00100000:0.0:1789760737.021552:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718259713944 time 718296832345 state 0x2 ref 0 00000100:00100000:3.0:1789760737.025230:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.025303:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229696 00000100:00100000:0.0:1789760737.025311:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229696 00000100:00100000:0.0:1789760737.025382:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718259713944 time 718296832345 state 0x0 ref 1 00000100:00100000:0.0:1789760737.025398:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718297092761 time 718296832345 state 0x1 ref 1 00000100:00100000:0.0:1789760737.025428:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718297092761 time 718340731160 state 0x0 ref 1 00000100:00100000:0.0:1789760737.025441:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507229696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.025487:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19185664, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0x9807 00010000:00010000:0.0:1789760737.025497:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.025527:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.025532:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.025538:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.025553:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760737.026048:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760737.050881:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19185664 -> 19189760 00002000:00010000:1.0:1789760737.050890:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:1.0:1789760737.050893:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:1.0:1789760737.050905:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050918:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050926:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050933:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.050938:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050946:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050965:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.050973:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7c4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.051064:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507229696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25621us (25838us total) trans 4294972004 rc 0/0 00000100:00100000:1.0:1789760737.051074:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4714 00000100:00100000:1.0:1789760737.051084:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718297092761 time 718340731160 state 0x2 ref 0 00000001:00080000:2.0:1789760737.051242:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972004 is committed 00000100:00100000:2.0:1789760737.055886:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.055964:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229824 00000100:00100000:1.0:1789760737.055974:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229824 00000100:00100000:1.0:1789760737.056026:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718297092761 time 718340731160 state 0x0 ref 1 00000100:00100000:1.0:1789760737.056042:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718340991576 time 718340731160 state 0x1 ref 1 00000100:00100000:1.0:1789760737.056073:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718340991576 time 718371376257 state 0x0 ref 1 00000100:00100000:1.0:1789760737.056087:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.056134:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19189760, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.056143:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.056158:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.056162:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.056169:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.056181:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.087738:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972005 is committed 00002000:00010000:1.0:1789760737.087988:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19189760 -> 19193856 00010000:00010000:1.0:1789760737.088005:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088018:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088027:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088035:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.088040:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088049:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088068:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.088075:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.088176:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507229824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32079us (32287us total) trans 4294972005 rc 0/0 00000100:00100000:1.0:1789760737.088189:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4715 00000100:00100000:1.0:1789760737.088198:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718340991576 time 718371376257 state 0x2 ref 0 00000100:00100000:1.0:1789760737.092272:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.094144:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507229952 00000100:00100000:0.0:1789760737.094158:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507229952 00000100:00100000:0.0:1789760737.094216:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718340991576 time 718371376257 state 0x0 ref 1 00000100:00100000:0.0:1789760737.094232:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718371636673 time 718371376257 state 0x1 ref 1 00000100:00100000:0.0:1789760737.094254:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718371636673 time 718409557303 state 0x0 ref 1 00000100:00100000:0.0:1789760737.094267:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507229952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.094312:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19193856, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.094325:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.094340:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.094344:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.094350:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.094363:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.125758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972006 is committed 00002000:00010000:0.0:1789760737.125849:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19193856 -> 19197952 00010000:00010000:0.0:1789760737.125921:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125935:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125945:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125952:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760737.125957:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125967:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125987:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.125994:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.126168:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507229952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31897us (33899us total) trans 4294972006 rc 0/0 00000100:00100000:0.0:1789760737.126179:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4716 00000100:00100000:0.0:1789760737.126188:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718371636673 time 718409557303 state 0x2 ref 0 00000100:00100000:3.0:1789760737.128976:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.129205:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230080 00000100:00100000:1.0:1789760737.129215:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230080 00000100:00100000:1.0:1789760737.129265:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718371636673 time 718409557303 state 0x0 ref 1 00000100:00100000:1.0:1789760737.129280:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718409817719 time 718409557303 state 0x1 ref 1 00000100:00100000:1.0:1789760737.129302:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718409817719 time 718444606218 state 0x0 ref 1 00000100:00100000:1.0:1789760737.129314:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507230080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.129356:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19197952, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.129365:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.129381:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.129385:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.129391:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.129405:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760737.171935:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19197952 -> 19202048 00010000:00010000:0.0:1789760737.171956:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.171969:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.171978:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.171986:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760737.171992:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.172001:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.172022:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.172030:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7c64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.172127:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507230080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42810us (43155us total) trans 4294972007 rc 0/0 00000100:00100000:0.0:1789760737.172138:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4717 00000100:00100000:0.0:1789760737.172148:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718409817719 time 718444606218 state 0x2 ref 0 00000001:00080000:2.0:1789760737.172285:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972007 is committed 00010000:00010000:2.0:1789760737.180043:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760737.180482:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.180583:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230208 00000100:00100000:1.0:1789760737.180594:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230208 00000100:00100000:1.0:1789760737.180649:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718409817719 time 718444606218 state 0x0 ref 1 00000100:00100000:1.0:1789760737.180663:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718444866634 time 718444606218 state 0x1 ref 1 00000100:00100000:1.0:1789760737.180707:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718444866634 time 718495996699 state 0x0 ref 1 00000100:00100000:1.0:1789760737.180719:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507230208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.180766:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19202048, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.180775:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.180790:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.180794:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.180801:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.180815:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.218325:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972008 is committed 00002000:00010000:3.0:1789760737.218571:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19202048 -> 19206144 00010000:00010000:3.0:1789760737.218586:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218597:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218606:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218613:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760737.218618:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218627:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218644:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.218653:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7c6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.218770:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507230208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38048us (38290us total) trans 4294972008 rc 0/0 00000100:00100000:3.0:1789760737.218779:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4718 00000100:00100000:3.0:1789760737.218789:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718444866634 time 718495996699 state 0x2 ref 0 00000100:00100000:3.0:1789760737.224574:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.224739:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230336 00000100:00100000:0.0:1789760737.224749:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230336 00000100:00100000:0.0:1789760737.224803:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718444866634 time 718495996699 state 0x0 ref 1 00000100:00100000:0.0:1789760737.224831:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718496257115 time 718495996699 state 0x1 ref 1 00000100:00100000:0.0:1789760737.224860:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718496257115 time 718540163814 state 0x0 ref 1 00000100:00100000:0.0:1789760737.224872:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.224924:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19206144, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.224934:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.224949:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.224953:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.224960:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.224973:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.278026:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972009 is committed 00002000:00010000:0.0:1789760737.278138:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19206144 -> 19210240 00010000:00010000:0.0:1789760737.278153:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278166:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278174:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278192:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760737.278198:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278208:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278232:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.278240:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.278338:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53456us (53762us total) trans 4294972009 rc 0/0 00000100:00100000:0.0:1789760737.278347:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4719 00000100:00100000:0.0:1789760737.278356:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718496257115 time 718540163814 state 0x2 ref 0 00000100:00100000:2.0:1789760737.284792:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.284896:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230464 00000100:00100000:0.0:1789760737.284906:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230464 00000100:00100000:0.0:1789760737.284959:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718496257115 time 718540163814 state 0x0 ref 1 00000100:00100000:0.0:1789760737.284975:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718540424230 time 718540163814 state 0x1 ref 1 00000100:00100000:0.0:1789760737.285008:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718540424230 time 718600310918 state 0x0 ref 1 00000100:00100000:0.0:1789760737.285022:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507230464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.285070:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19210240, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.285080:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.285096:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.285101:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.285107:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.285121:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.321945:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972010 is committed 00002000:00010000:0.0:1789760737.322132:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19210240 -> 19214336 00010000:00010000:0.0:1789760737.322148:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322160:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322169:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322176:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760737.322192:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322203:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322224:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.322236:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.322463:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507230464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37438us (37675us total) trans 4294972010 rc 0/0 00000100:00100000:0.0:1789760737.322475:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4720 00000100:00100000:0.0:1789760737.322495:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718540424230 time 718600310918 state 0x2 ref 0 00000100:00100000:2.0:1789760737.327834:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760737.328256:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230592 00000100:00100000:3.0:1789760737.328286:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230592 00000100:00100000:3.0:1789760737.328367:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718540424230 time 718600310918 state 0x0 ref 1 00000100:00100000:3.0:1789760737.328395:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718600571334 time 718600310918 state 0x1 ref 1 00000100:00100000:3.0:1789760737.328434:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718600571334 time 718643737358 state 0x0 ref 1 00000100:00100000:3.0:1789760737.328449:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760737.328515:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19214336, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:3.0:1789760737.328526:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.328545:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760737.328551:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.328559:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.328576:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.397281:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972011 is committed 00002000:00010000:2.0:1789760737.397342:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19214336 -> 19218432 00010000:00010000:2.0:1789760737.397357:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397370:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397380:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397387:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760737.397392:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397401:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397421:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.397429:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7c80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.397531:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69081us (69711us total) trans 4294972011 rc 0/0 00000100:00100000:2.0:1789760737.397542:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4721 00000100:00100000:2.0:1789760737.397552:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718600571334 time 718643737358 state 0x2 ref 0 00000100:00100000:3.0:1789760737.401360:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.401467:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230720 00000100:00100000:0.0:1789760737.401478:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230720 00000100:00100000:0.0:1789760737.401533:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718600571334 time 718643737358 state 0x0 ref 1 00000100:00100000:0.0:1789760737.401549:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718643997774 time 718643737358 state 0x1 ref 1 00000100:00100000:0.0:1789760737.401574:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718643997774 time 718716877230 state 0x0 ref 1 00000100:00100000:0.0:1789760737.401585:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507230720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.401633:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19218432, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.401642:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.401659:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.401663:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.401671:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.401689:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.433581:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972012 is committed 00002000:00010000:2.0:1789760737.433783:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19218432 -> 19222528 00010000:00010000:2.0:1789760737.433799:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433812:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433838:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433845:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760737.433850:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433867:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433887:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.433895:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7c87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.433988:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507230720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32398us (32631us total) trans 4294972012 rc 0/0 00000100:00100000:2.0:1789760737.433998:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4722 00000100:00100000:2.0:1789760737.434008:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718643997774 time 718716877230 state 0x2 ref 0 00000100:00100000:3.0:1789760737.436925:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.437049:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230848 00000100:00100000:1.0:1789760737.437059:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230848 00000100:00100000:1.0:1789760737.437110:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718643997774 time 718716877230 state 0x0 ref 1 00000100:00100000:1.0:1789760737.437126:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718717137646 time 718716877230 state 0x1 ref 1 00000100:00100000:1.0:1789760737.437156:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718717137646 time 718752459994 state 0x0 ref 1 00000100:00100000:1.0:1789760737.437172:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.437228:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19222528, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.437237:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.437253:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.437257:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.437264:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.437279:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.466317:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19222528 -> 19226624 00010000:00010000:3.0:1789760737.466337:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466348:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466357:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466364:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760737.466369:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466377:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466395:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.466402:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b7c8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.466493:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507230848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29317us (29570us total) trans 4294972013 rc 0/0 00000100:00100000:3.0:1789760737.466503:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4723 00000100:00100000:3.0:1789760737.466521:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718717137646 time 718752459994 state 0x2 ref 0 00000001:00080000:3.0:1789760737.466565:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972013 is committed 00000100:00100000:0.0:1789760737.478008:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760737.478225:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507230976 00000100:00100000:2.0:1789760737.478236:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507230976 00000100:00100000:2.0:1789760737.478357:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718717137646 time 718752459994 state 0x0 ref 1 00000100:00100000:2.0:1789760737.478372:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718752720410 time 718752459994 state 0x1 ref 1 00000100:00100000:2.0:1789760737.478403:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718752720410 time 718793706376 state 0x0 ref 1 00000100:00100000:2.0:1789760737.478415:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507230976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760737.478462:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19226624, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:2.0:1789760737.478470:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.478587:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760737.478593:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.478600:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.478615:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760737.504610:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19226624 -> 19230720 00010000:00010000:1.0:1789760737.504630:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504643:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504652:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504660:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.504665:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504675:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504715:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.504735:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7c95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.504832:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507230976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26403us (26818us total) trans 4294972014 rc 0/0 00000100:00100000:1.0:1789760737.504842:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4724 00000100:00100000:1.0:1789760737.504851:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718752720410 time 718793706376 state 0x2 ref 0 00000001:00080000:2.0:1789760737.505032:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972014 is committed 00000100:00100000:1.0:1789760737.513362:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760737.513466:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231104 00000100:00100000:2.0:1789760737.513476:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231104 00000100:00100000:2.0:1789760737.513602:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718752720410 time 718793706376 state 0x0 ref 1 00000100:00100000:2.0:1789760737.513618:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718793966792 time 718793706376 state 0x1 ref 1 00000100:00100000:2.0:1789760737.513650:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718793966792 time 718828953338 state 0x0 ref 1 00000100:00100000:2.0:1789760737.513662:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507231104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.517358:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19230720, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.517374:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.517391:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760737.517396:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.517402:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.517417:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.579295:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972015 is committed 00002000:00010000:1.0:1789760737.579442:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19230720 -> 19234816 00010000:00010000:1.0:1789760737.579458:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579470:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579479:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579559:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.579564:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579573:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579592:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.579600:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7c9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.580324:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507231104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66028us (66336us total) trans 4294972015 rc 0/0 00000100:00100000:1.0:1789760737.580339:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4725 00000100:00100000:1.0:1789760737.580349:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718793966792 time 718828953338 state 0x2 ref 0 00000100:00100000:3.0:1789760737.591790:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.591941:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231232 00000100:00100000:1.0:1789760737.591950:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231232 00000100:00100000:1.0:1789760737.591998:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718793966792 time 718828953338 state 0x0 ref 1 00000100:00100000:1.0:1789760737.592012:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718829213754 time 718828953338 state 0x1 ref 1 00000100:00100000:1.0:1789760737.592039:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718829213754 time 718907342689 state 0x0 ref 1 00000100:00100000:1.0:1789760737.592054:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.592100:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19234816, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.592108:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.592124:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.592128:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.592135:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.592149:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.651959:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972016 is committed 00002000:00010000:1.0:1789760737.652058:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19234816 -> 19238912 00010000:00010000:1.0:1789760737.652075:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652087:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652096:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652102:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.652107:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652116:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652134:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.652142:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7ca3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.652268:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60210us (60482us total) trans 4294972016 rc 0/0 00000100:00100000:1.0:1789760737.652278:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4726 00000100:00100000:1.0:1789760737.652288:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718829213754 time 718907342689 state 0x2 ref 0 00000100:00100000:0.0:1789760737.661247:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.661480:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231360 00000100:00100000:1.0:1789760737.661490:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231360 00000100:00100000:1.0:1789760737.661546:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718829213754 time 718907342689 state 0x0 ref 1 00000100:00100000:1.0:1789760737.661563:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718907603105 time 718907342689 state 0x1 ref 1 00000100:00100000:1.0:1789760737.661592:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718907603105 time 718976895213 state 0x0 ref 1 00000100:00100000:1.0:1789760737.661604:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.661656:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19238912, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.661664:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.661680:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760737.661685:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.661830:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.661953:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.690366:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972017 is committed 00002000:00010000:1.0:1789760737.690540:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19238912 -> 19243008 00010000:00010000:1.0:1789760737.690556:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690570:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690580:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690586:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.690591:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690599:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690620:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.690628:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7caa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.690794:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29186us (29551us total) trans 4294972017 rc 0/0 00000100:00100000:1.0:1789760737.690804:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4727 00000100:00100000:1.0:1789760737.690814:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718907603105 time 718976895213 state 0x2 ref 0 00000100:00100000:2.0:1789760737.698900:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.699056:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231488 00000100:00100000:1.0:1789760737.699068:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231488 00000100:00100000:1.0:1789760737.699125:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718907603105 time 718976895213 state 0x0 ref 1 00000100:00100000:1.0:1789760737.699139:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718977155629 time 718976895213 state 0x1 ref 1 00000100:00100000:1.0:1789760737.699179:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718977155629 time 719014482518 state 0x0 ref 1 00000100:00100000:1.0:1789760737.699191:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.699239:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19243008, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.699248:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.699264:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760737.699269:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.699276:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.699296:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.742389:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972018 is committed 00002000:00010000:2.0:1789760737.742493:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19243008 -> 19247104 00010000:00010000:2.0:1789760737.742508:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742521:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742531:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742538:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760737.742543:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742553:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742573:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.742581:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7cb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.742677:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507231488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43482us (43783us total) trans 4294972018 rc 0/0 00000100:00100000:2.0:1789760737.742687:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4728 00000100:00100000:2.0:1789760737.742745:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718977155629 time 719014482518 state 0x2 ref 0 00000100:00100000:0.0:1789760737.746844:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.747249:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231616 00000100:00100000:1.0:1789760737.747332:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231616 00000100:00100000:1.0:1789760737.747391:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 718977155629 time 719014482518 state 0x0 ref 1 00000100:00100000:1.0:1789760737.747406:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719014742934 time 719014482518 state 0x1 ref 1 00000100:00100000:1.0:1789760737.747438:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719014742934 time 719062740960 state 0x0 ref 1 00000100:00100000:1.0:1789760737.747450:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507231616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.747500:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19247104, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.747509:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.747524:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760737.747529:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.747535:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.747550:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.790783:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972019 is committed 00002000:00010000:2.0:1789760737.791001:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19247104 -> 19251200 00010000:00010000:2.0:1789760737.791018:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791032:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791047:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791054:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760737.791058:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791067:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791099:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.791112:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7cb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.791202:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507231616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43749us (44379us total) trans 4294972019 rc 0/0 00000100:00100000:2.0:1789760737.791212:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4729 00000100:00100000:2.0:1789760737.791221:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719014742934 time 719062740960 state 0x2 ref 0 00000100:00100000:0.0:1789760737.795511:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.795668:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231744 00000100:00100000:1.0:1789760737.795680:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231744 00000100:00100000:1.0:1789760737.795759:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719014742934 time 719062740960 state 0x0 ref 1 00000100:00100000:1.0:1789760737.795774:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719063001376 time 719062740960 state 0x1 ref 1 00000100:00100000:1.0:1789760737.795803:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719063001376 time 719111105859 state 0x0 ref 1 00000100:00100000:1.0:1789760737.795826:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507231744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.795928:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19251200, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.795939:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.795954:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760737.795959:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.795966:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.795980:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.829347:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972020 is committed 00002000:00010000:1.0:1789760737.829732:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19251200 -> 19255296 00010000:00010000:1.0:1789760737.829750:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829762:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829776:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829783:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.829787:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829795:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829975:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.829986:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.830115:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507231744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34297us (34609us total) trans 4294972020 rc 0/0 00000100:00100000:1.0:1789760737.830127:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4730 00000100:00100000:1.0:1789760737.830136:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719063001376 time 719111105859 state 0x2 ref 0 00000100:00100000:1.0:1789760737.835676:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760737.836185:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507231872 00000100:00100000:2.0:1789760737.836195:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507231872 00000100:00100000:2.0:1789760737.836339:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719063001376 time 719111105859 state 0x0 ref 1 00000100:00100000:2.0:1789760737.836355:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719111366275 time 719111105859 state 0x1 ref 1 00000100:00100000:2.0:1789760737.836385:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719111366275 time 719151687787 state 0x0 ref 1 00000100:00100000:2.0:1789760737.836399:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507231872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760737.836449:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19255296, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:2.0:1789760737.836459:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.836475:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760737.836480:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.836486:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.836506:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.894564:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19255296 -> 19259392 00010000:00010000:2.0:1789760737.894582:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894594:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894603:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894609:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760737.894614:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894622:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894641:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.894648:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7cc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.894768:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507231872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58369us (59098us total) trans 4294972021 rc 0/0 00000100:00100000:2.0:1789760737.894778:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4731 00000100:00100000:2.0:1789760737.894786:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719111366275 time 719151687787 state 0x2 ref 0 00000001:00080000:3.0:1789760737.895198:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972021 is committed 00000100:00100000:1.0:1789760737.897671:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760737.898028:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232000 00000100:00100000:2.0:1789760737.898039:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232000 00000100:00100000:2.0:1789760737.898095:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719111366275 time 719151687787 state 0x0 ref 1 00000100:00100000:2.0:1789760737.898111:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719151948203 time 719151687787 state 0x1 ref 1 00000100:00100000:2.0:1789760737.898180:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719151948203 time 719213483083 state 0x0 ref 1 00000100:00100000:2.0:1789760737.898194:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760737.898237:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19259392, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:2.0:1789760737.898246:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.898261:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760737.898265:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.898271:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.898285:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.928638:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972022 is committed 00002000:00010000:1.0:1789760737.928983:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19259392 -> 19263488 00010000:00010000:1.0:1789760737.928997:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929010:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929018:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929025:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.929030:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929038:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929065:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.929073:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7ccd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.929160:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30964us (31494us total) trans 4294972022 rc 0/0 00000100:00100000:1.0:1789760737.929169:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4732 00000100:00100000:1.0:1789760737.929178:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719151948203 time 719213483083 state 0x2 ref 0 00000100:00100000:2.0:1789760737.934410:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760737.934727:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232128 00000100:00100000:1.0:1789760737.934737:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232128 00000100:00100000:1.0:1789760737.934794:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719151948203 time 719213483083 state 0x0 ref 1 00000100:00100000:1.0:1789760737.934810:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719213743499 time 719213483083 state 0x1 ref 1 00000100:00100000:1.0:1789760737.934918:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719213743499 time 719250219602 state 0x0 ref 1 00000100:00100000:1.0:1789760737.934931:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760737.934979:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19263488, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760737.934989:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.935004:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760737.935009:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.935015:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.935030:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.972160:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972023 is committed 00002000:00010000:1.0:1789760737.972239:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19263488 -> 19267584 00010000:00010000:1.0:1789760737.972254:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972267:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972281:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972288:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760737.972293:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972303:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972324:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.972332:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.972458:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37518us (38047us total) trans 4294972023 rc 0/0 00000100:00100000:1.0:1789760737.972468:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4733 00000100:00100000:1.0:1789760737.972478:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719213743499 time 719250219602 state 0x2 ref 0 00000100:00100000:3.0:1789760737.978060:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760737.978141:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232256 00000100:00100000:0.0:1789760737.978152:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232256 00000100:00100000:0.0:1789760737.978202:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719213743499 time 719250219602 state 0x0 ref 1 00000100:00100000:0.0:1789760737.978217:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719250480018 time 719250219602 state 0x1 ref 1 00000100:00100000:0.0:1789760737.978246:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719250480018 time 719293550301 state 0x0 ref 1 00000100:00100000:0.0:1789760737.978259:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507232256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760737.978302:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19267584, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:0.0:1789760737.978310:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.978324:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760737.978328:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.978333:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.978345:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.008726:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972024 is committed 00002000:00010000:0.0:1789760738.008865:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19267584 -> 19271680 00010000:00010000:0.0:1789760738.008881:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008894:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008905:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008912:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760738.008916:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008927:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008949:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.008956:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b7cdb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.009043:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507232256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30781us (30988us total) trans 4294972024 rc 0/0 00000100:00100000:0.0:1789760738.009052:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4734 00000100:00100000:0.0:1789760738.009074:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719250480018 time 719293550301 state 0x2 ref 0 00000100:00100000:3.0:1789760738.013233:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760738.013279:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.013354:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232384 00000100:00100000:2.0:1789760738.013364:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232384 00000100:00100000:2.0:1789760738.013420:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 466 00000100:00100000:2.0:1789760738.013430:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223497c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507232384:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760738.013501:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223497c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507232384:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 72us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760738.013508:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 466 00000100:00100000:3.0:1789760738.013909:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.013978:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232640 00000100:00100000:1.0:1789760738.013983:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232640 00000100:00100000:1.0:1789760738.014025:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719250480018 time 719293550301 state 0x0 ref 1 00000100:00100000:1.0:1789760738.014042:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719293810717 time 719293550301 state 0x1 ref 1 00000100:00100000:1.0:1789760738.014071:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719293810717 time 719329374379 state 0x0 ref 1 00000100:00100000:1.0:1789760738.014081:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.014126:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19271680, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0x9807 00010000:00010000:1.0:1789760738.014135:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.014151:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760738.014155:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.014161:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.014173:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.020721:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.023888:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760738.023991:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232512 00000100:00100000:0.0:1789760738.024001:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232512 00000100:00100000:0.0:1789760738.024068:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 878 00000100:00100000:0.0:1789760738.024080:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507232512:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760738.024154:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507232512:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 76us (10874us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760738.024163:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 878 00000100:00100000:0.0:1789760738.024183:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232768 00000100:00100000:0.0:1789760738.024186:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232768 00000100:00100000:0.0:1789760738.024212:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 879 00000100:00100000:0.0:1789760738.024219:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232768:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760738.024247:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232768:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 29us (3537us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760738.024253:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 879 00000100:00080000:3.0:1789760738.028039:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760738.028051:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760738.028470:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.030418:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233024 00000100:00100000:1.0:1789760738.030431:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233024 00000100:00080000:2.0:1789760738.030523:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760738.030562:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507232896 00000100:00100000:0.0:1789760738.030567:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507232896 00000100:00100000:0.0:1789760738.030620:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 880 00000100:00100000:0.0:1789760738.030630:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108042680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507233024:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760738.030727:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108042680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507233024:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 98us (6849us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760738.030735:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 880 00000100:00100000:0.0:1789760738.030751:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 881 00000100:00100000:0.0:1789760738.030759:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507232896:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760738.030786:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507232896:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 28us (2322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760738.030793:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 881 00000100:00080000:2.0:1789760738.030967:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760738.049075:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972025 is committed 00002000:00010000:1.0:1789760738.049282:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19271680 -> 19275776 00002000:00010000:1.0:1789760738.049288:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:1.0:1789760738.049290:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:1.0:1789760738.049304:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049318:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049327:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049335:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760738.049339:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049348:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049368:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.049376:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.049466:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507232640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35380us (35551us total) trans 4294972025 rc 0/0 00000100:00100000:1.0:1789760738.049476:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4735 00000100:00100000:1.0:1789760738.049486:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719293810717 time 719329374379 state 0x2 ref 0 00000100:00100000:2.0:1789760738.060345:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.061737:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233152 00000100:00100000:1.0:1789760738.061751:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233152 00000100:00100000:1.0:1789760738.061807:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719293810717 time 719329374379 state 0x0 ref 1 00000100:00100000:1.0:1789760738.061835:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719329634795 time 719329374379 state 0x1 ref 1 00000100:00100000:1.0:1789760738.061871:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719329634795 time 719377173901 state 0x0 ref 1 00000100:00100000:1.0:1789760738.061885:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507233152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.061934:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19275776, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:1.0:1789760738.061942:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.061957:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760738.061961:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.061967:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.061979:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.097706:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972026 is committed 00002000:00010000:1.0:1789760738.097799:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19275776 -> 19279872 00010000:00010000:1.0:1789760738.097823:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097836:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097845:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097859:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760738.097865:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097875:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097895:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.097903:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7ce9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.097996:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507233152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36109us (37658us total) trans 4294972026 rc 0/0 00000100:00100000:1.0:1789760738.098006:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4736 00000100:00100000:1.0:1789760738.098015:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719329634795 time 719377173901 state 0x2 ref 0 00000100:00100000:0.0:1789760738.107648:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.107783:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233280 00000100:00100000:2.0:1789760738.107794:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233280 00000100:00100000:2.0:1789760738.107860:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719329634795 time 719377173901 state 0x0 ref 1 00000100:00100000:2.0:1789760738.107876:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719377434317 time 719377173901 state 0x1 ref 1 00000100:00100000:2.0:1789760738.107955:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719377434317 time 719423257024 state 0x0 ref 1 00000100:00100000:2.0:1789760738.107969:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507233280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.108015:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19279872, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.108025:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.108040:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.108045:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.108051:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.108065:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.171754:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19279872 -> 19283968 00010000:00010000:1.0:1789760738.171775:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171788:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171798:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171811:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760738.171827:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171837:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171857:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.171866:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7cf0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.171963:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507233280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63992us (64321us total) trans 4294972027 rc 0/0 00000100:00100000:1.0:1789760738.171974:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4737 00000100:00100000:1.0:1789760738.171983:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719377434317 time 719423257024 state 0x2 ref 0 00000001:00080000:1.0:1789760738.172939:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972027 is committed 00000100:00100000:0.0:1789760738.182291:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.182390:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233408 00000100:00100000:1.0:1789760738.182400:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233408 00000100:00100000:1.0:1789760738.182454:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719377434317 time 719423257024 state 0x0 ref 1 00000100:00100000:1.0:1789760738.182469:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719423517440 time 719423257024 state 0x1 ref 1 00000100:00100000:1.0:1789760738.182535:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719423517440 time 719497837990 state 0x0 ref 1 00000100:00100000:1.0:1789760738.182549:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507233408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.182595:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19283968, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:1.0:1789760738.182604:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.182621:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760738.182626:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.182633:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.182646:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203899:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760738.263283:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972028 is committed 00002000:00010000:2.0:1789760738.263423:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19283968 -> 19288064 00010000:00010000:2.0:1789760738.263440:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263454:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263464:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263471:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.263476:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263486:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263509:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.263518:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b7cf7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.263613:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507233408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 81062us (81327us total) trans 4294972028 rc 0/0 00000100:00100000:2.0:1789760738.263624:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4738 00000100:00100000:2.0:1789760738.263633:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719423517440 time 719497837990 state 0x2 ref 0 00000100:00100000:3.0:1789760738.267463:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.267921:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233536 00000100:00100000:1.0:1789760738.267932:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233536 00000100:00100000:1.0:1789760738.267987:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719423517440 time 719497837990 state 0x0 ref 1 00000100:00100000:1.0:1789760738.268003:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719498098406 time 719497837990 state 0x1 ref 1 00000100:00100000:1.0:1789760738.268038:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719498098406 time 719583340762 state 0x0 ref 1 00000100:00100000:1.0:1789760738.268052:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507233536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.268103:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19288064, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:1.0:1789760738.268112:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.268133:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760738.268138:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.268144:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.268220:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.311659:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19288064 -> 19292160 00010000:00010000:2.0:1789760738.311678:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311723:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311735:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311742:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.311747:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311761:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311782:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.311790:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7cfe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.311998:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507233536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43943us (44538us total) trans 4294972029 rc 0/0 00000100:00100000:2.0:1789760738.312011:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4739 00000100:00100000:2.0:1789760738.312020:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719498098406 time 719583340762 state 0x2 ref 0 00000001:00080000:3.0:1789760738.313021:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972029 is committed 00000100:00100000:3.0:1789760738.317336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760738.317872:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233664 00000100:00100000:1.0:1789760738.317882:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233664 00000100:00100000:1.0:1789760738.317942:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719498098406 time 719583340762 state 0x0 ref 1 00000100:00100000:1.0:1789760738.317961:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719583601178 time 719583340762 state 0x1 ref 1 00000100:00100000:1.0:1789760738.317998:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719583601178 time 719633301380 state 0x0 ref 1 00000100:00100000:1.0:1789760738.318012:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.318069:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19292160, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:1.0:1789760738.318078:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.318094:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760738.318099:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.318105:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.318125:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.344221:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972030 is committed 00002000:00010000:2.0:1789760738.344297:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19292160 -> 19296256 00010000:00010000:2.0:1789760738.344312:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344325:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344334:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344341:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.344346:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344355:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344371:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.344379:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b7d05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.344458:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26444us (27127us total) trans 4294972030 rc 0/0 00000100:00100000:2.0:1789760738.344468:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4740 00000100:00100000:2.0:1789760738.344477:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719583601178 time 719633301380 state 0x2 ref 0 00000100:00100000:1.0:1789760738.347902:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.348092:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233792 00000100:00100000:2.0:1789760738.348103:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233792 00000100:00100000:2.0:1789760738.348225:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719583601178 time 719633301380 state 0x0 ref 1 00000100:00100000:2.0:1789760738.348241:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719633561796 time 719633301380 state 0x1 ref 1 00000100:00100000:2.0:1789760738.348271:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719633561796 time 719663575071 state 0x0 ref 1 00000100:00100000:2.0:1789760738.348284:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.348333:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19296256, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.348343:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.348357:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.348361:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.348368:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.348381:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.388331:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972031 is committed 00002000:00010000:2.0:1789760738.388401:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19296256 -> 19300352 00010000:00010000:2.0:1789760738.388418:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388430:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388439:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388445:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.388449:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388458:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388481:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.388489:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19296256->19300351] (req 19296256->19300351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.388606:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40319us (40707us total) trans 4294972031 rc 0/0 00000100:00100000:2.0:1789760738.388615:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4741 00000100:00100000:2.0:1789760738.388624:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719633561796 time 719663575071 state 0x2 ref 0 00000100:00100000:3.0:1789760738.393050:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.393206:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507233920 00000100:00100000:2.0:1789760738.393216:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507233920 00000100:00100000:2.0:1789760738.393278:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719633561796 time 719663575071 state 0x0 ref 1 00000100:00100000:2.0:1789760738.393297:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719663835487 time 719663575071 state 0x1 ref 1 00000100:00100000:2.0:1789760738.393317:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719663835487 time 719708621244 state 0x0 ref 1 00000100:00100000:2.0:1789760738.393329:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.393374:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19300352, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.393382:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.393397:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.393403:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.393409:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.393422:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.422671:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972032 is committed 00002000:00010000:2.0:1789760738.422719:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19300352 -> 19304448 00010000:00010000:2.0:1789760738.422735:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422747:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422757:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422764:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.422769:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422778:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422796:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.422803:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.422965:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507233920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29632us (29919us total) trans 4294972032 rc 0/0 00000100:00100000:2.0:1789760738.422975:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4742 00000100:00100000:2.0:1789760738.422985:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719663835487 time 719708621244 state 0x2 ref 0 00000100:00100000:3.0:1789760738.426298:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.426441:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234048 00000100:00100000:2.0:1789760738.426451:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234048 00000100:00100000:2.0:1789760738.426503:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719663835487 time 719708621244 state 0x0 ref 1 00000100:00100000:2.0:1789760738.426518:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719708881660 time 719708621244 state 0x1 ref 1 00000100:00100000:2.0:1789760738.426549:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719708881660 time 719741852346 state 0x0 ref 1 00000100:00100000:2.0:1789760738.426562:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507234048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.426608:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19304448, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.426616:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.426630:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.426635:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.426641:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.426652:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760738.467551:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19304448 -> 19308544 00010000:00010000:0.0:1789760738.467574:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467587:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467602:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467609:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760738.467614:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467624:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467643:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.467651:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b7d1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.467789:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507234048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41211us (41482us total) trans 4294972033 rc 0/0 00000100:00100000:0.0:1789760738.467800:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4743 00000100:00100000:0.0:1789760738.468019:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719708881660 time 719741852346 state 0x2 ref 0 00000001:00080000:1.0:1789760738.468907:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972033 is committed 00000100:00100000:1.0:1789760738.477761:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.477867:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234176 00000100:00100000:2.0:1789760738.477877:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234176 00000100:00100000:2.0:1789760738.477928:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719708881660 time 719741852346 state 0x0 ref 1 00000100:00100000:2.0:1789760738.477943:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719742112762 time 719741852346 state 0x1 ref 1 00000100:00100000:2.0:1789760738.477962:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719742112762 time 719793266275 state 0x0 ref 1 00000100:00100000:2.0:1789760738.477974:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507234176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.478030:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19308544, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.478040:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.478054:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.478058:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.478065:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.478079:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.525791:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b45aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538271616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.525874:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.525923:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538272128:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.525950:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.525970:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538272640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526007:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526026:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538273152:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526047:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526071:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:3.0:1789760738.526077:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538271488:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760738.526097:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526112:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538272000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526132:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526147:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538272512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526178:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526194:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538273024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526211:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.526241:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271616 00000100:00100000:3.0:1789760738.526248:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271616 00000100:00100000:3.0:1789760738.526681:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760738.526720:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223495c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538271616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526777:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223495c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538271616:12345-0@lo:400:kworker.0 Request processed in 87us (903us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.526784:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760738.526827:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272128 00000100:00100000:3.0:1789760738.526831:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272128 00000100:00100000:3.0:1789760738.526865:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789760738.526871:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538272128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.526904:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538272128:12345-0@lo:400:kworker.0 Request processed in 33us (953us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.526913:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789760738.526929:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272640 00000100:00100000:3.0:1789760738.526934:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272640 00000100:00100000:3.0:1789760738.526964:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273152 00000100:00100000:3.0:1789760738.526967:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273152 00000100:00100000:3.0:1789760738.526993:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272512 00000100:00100000:3.0:1789760738.526996:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272512 00000100:00100000:3.0:1789760738.527010:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273024 00000100:00100000:3.0:1789760738.527012:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273024 00000100:00100000:3.0:1789760738.527024:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1789760738.527030:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538272640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527062:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538272640:12345-0@lo:400:kworker.0 Request processed in 32us (1055us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527068:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1789760738.527078:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789760738.527083:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538273152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527108:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538273152:12345-0@lo:400:kworker.0 Request processed in 25us (1060us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527114:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789760738.527124:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789760738.527129:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538272512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527362:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538272512:12345-0@lo:400:kworker.0 Request processed in 26us (976us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527371:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789760738.527384:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:3.0:1789760738.527390:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538273024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527423:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538273024:12345-0@lo:400:kworker.0 Request processed in 33us (1211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527428:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:3.0:1789760738.527451:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271488 00000100:00100000:3.0:1789760738.527454:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271488 00000100:00100000:3.0:1789760738.527546:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1789760738.527554:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108107b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538271488:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760738.527582:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760738.527616:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108107b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538271488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (1517us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527621:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1789760738.527639:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272000 00000100:00100000:3.0:1789760738.527643:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272000 00000100:00100000:3.0:1789760738.527665:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 882 00000100:00100000:3.0:1789760738.527671:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538272000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527729:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538272000:12345-0@lo:400:kworker.0 Request processed in 57us (1595us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.527734:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 882 00000100:00100000:3.0:1789760738.527787:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538271488:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760738.527799:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538272000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527820:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538272512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527831:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538273024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.527840:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b45aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538271616:0@lo:400:kworker.0 00000001:00080000:0.0:1789760738.529898:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972034 is committed 00000100:00100000:3.0:1789760738.530114:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538272128:0@lo:400:kworker.0 00000100:00080000:3.0:1789760738.530134:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760738.530139:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760738.530331:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108040700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271360:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760738.530400:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760738.530433:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271872:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.530458:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.530568:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538272640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.530581:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538273152:0@lo:400:kworker.0 00002000:00010000:2.0:1789760738.530775:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19308544 -> 19312640 00010000:00010000:2.0:1789760738.530791:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.530806:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.530824:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.530830:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.530834:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.530842:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.530964:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538271744:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.531150:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538272384:0@lo:400:kworker.0 00010000:00010000:2.0:1789760738.531246:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.531355:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760738.531385:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538272256:0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.531406:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760738.531424:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538272768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.531444:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.531568:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271744 00000100:00100000:3.0:1789760738.531575:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271744 00000100:00100000:3.0:1789760738.531613:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272256 00000100:00100000:3.0:1789760738.531616:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272256 00000100:00100000:3.0:1789760738.531642:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272768 00000100:00100000:3.0:1789760738.531645:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272768 00000100:00100000:3.0:1789760738.531662:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 883 00000100:00100000:3.0:1789760738.531671:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538271744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.531745:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538271744:12345-0@lo:400:kworker.0 Request processed in 74us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.531753:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 883 00000100:00100000:3.0:1789760738.531770:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 884 00000100:00100000:3.0:1789760738.531777:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538272256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.531806:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538272256:12345-0@lo:400:kworker.0 Request processed in 30us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.531824:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 884 00000100:00100000:3.0:1789760738.531837:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 885 00000100:00100000:3.0:1789760738.531843:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538272768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.531884:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538272768:12345-0@lo:400:kworker.0 Request processed in 41us (439us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.531891:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 885 00000100:00080000:3.0:1789760738.531908:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760738.532064:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760738.532094:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538272896:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.532120:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789760738.532328:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.532403:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507234176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54428us (54650us total) trans 4294972034 rc 0/0 00000100:00100000:2.0:1789760738.532413:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4744 00000100:00100000:2.0:1789760738.532422:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719742112762 time 719793266275 state 0x2 ref 0 00000100:00100000:1.0:1789760738.532547:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272384 00000100:00100000:1.0:1789760738.532552:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272384 00000100:00100000:1.0:1789760738.532590:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538272896 00000100:00100000:1.0:1789760738.532593:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538272896 00000100:00100000:1.0:1789760738.532608:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 886 00000100:00100000:1.0:1789760738.532614:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538272384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.532655:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538272384:12345-0@lo:400:kworker.0 Request processed in 40us (591us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760738.532661:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 886 00000100:00100000:1.0:1789760738.532673:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 887 00000100:00100000:1.0:1789760738.532678:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538272896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.532723:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538272896:12345-0@lo:400:kworker.0 Request processed in 44us (602us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760738.532729:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 887 00000100:00100000:1.0:1789760738.532766:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538271744:0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.532777:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538272256:0@lo:400:kworker.0 00000100:00100000:1.0:1789760738.532786:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538272768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.532963:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271360 00000100:00100000:0.0:1789760738.532969:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271360 00000100:00100000:0.0:1789760738.533088:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1789760738.533096:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538271360:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760738.533113:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760738.533152:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538271360:12345-0@lo:13:osp-pre-1-1.0 Request processed in 57us (2751us total) trans 0 rc 0/0 00000100:00080000:2.0:1789760738.533340:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760738.533721:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1789760738.533742:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538271872 00000100:00100000:0.0:1789760738.533747:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538271872 00000100:00100000:0.0:1789760738.533779:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1789760738.533785:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538271872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.533833:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538271872:12345-0@lo:400:kworker.0 Request processed in 48us (3374us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760738.533839:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1789760738.533888:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108040700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271360:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760738.533899:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538271872:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.533910:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538272384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760738.533919:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538272896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.538629:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.538762:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234304 00000100:00100000:2.0:1789760738.538771:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234304 00000100:00100000:2.0:1789760738.538830:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719742112762 time 719793266275 state 0x0 ref 1 00000100:00100000:2.0:1789760738.538844:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719793526691 time 719793266275 state 0x1 ref 1 00000100:00100000:2.0:1789760738.538876:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719793526691 time 719854179477 state 0x0 ref 1 00000100:00100000:2.0:1789760738.538887:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507234304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.538929:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19312640, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.538938:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.538952:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.538957:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.538962:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.538985:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.581117:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972035 is committed 00002000:00010000:2.0:1789760738.581226:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19312640 -> 19316736 00010000:00010000:2.0:1789760738.581241:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581363:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581378:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581385:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.581390:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581399:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581481:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.581490:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.581583:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507234304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42692us (42958us total) trans 4294972035 rc 0/0 00000100:00100000:2.0:1789760738.581593:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4745 00000100:00100000:2.0:1789760738.581602:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719793526691 time 719854179477 state 0x2 ref 0 00000100:00100000:3.0:1789760738.586283:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760738.586430:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234432 00000100:00100000:0.0:1789760738.586442:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234432 00000100:00100000:0.0:1789760738.587508:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719793526691 time 719854179477 state 0x0 ref 1 00000100:00100000:0.0:1789760738.587526:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719854439893 time 719854179477 state 0x1 ref 1 00000100:00100000:2.0:1789760738.588438:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719854439893 time 719903735818 state 0x0 ref 1 00000100:00100000:2.0:1789760738.588466:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.588524:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19316736, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.588534:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.588550:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.588555:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.588561:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.588576:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.641689:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19316736 -> 19320832 00010000:00010000:2.0:1789760738.641725:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641738:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641747:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641754:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.641759:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641767:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641786:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.641794:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.641952:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507234432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53495us (55676us total) trans 4294972036 rc 0/0 00000100:00100000:2.0:1789760738.641962:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4746 00000100:00100000:2.0:1789760738.641971:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719854439893 time 719903735818 state 0x2 ref 0 00000001:00080000:0.0:1789760738.642041:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972036 is committed 00000100:00100000:3.0:1789760738.646546:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.646687:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234560 00000100:00100000:2.0:1789760738.646732:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234560 00000100:00100000:2.0:1789760738.646784:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719854439893 time 719903735818 state 0x0 ref 1 00000100:00100000:2.0:1789760738.646798:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719903996234 time 719903735818 state 0x1 ref 1 00000100:00100000:2.0:1789760738.646831:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719903996234 time 719962134256 state 0x0 ref 1 00000100:00100000:2.0:1789760738.646842:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.646885:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19320832, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.646894:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.646909:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760738.646914:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.646920:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.646933:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.717667:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972037 is committed 00002000:00010000:2.0:1789760738.718162:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19320832 -> 19324928 00010000:00010000:2.0:1789760738.718178:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718191:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718201:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718208:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.718212:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718221:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718243:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.718256:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.718349:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71504us (71808us total) trans 4294972037 rc 0/0 00000100:00100000:2.0:1789760738.718359:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4747 00000100:00100000:2.0:1789760738.718368:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719903996234 time 719962134256 state 0x2 ref 0 00000100:00100000:3.0:1789760738.725276:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.725797:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234688 00000100:00100000:2.0:1789760738.725817:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234688 00000100:00100000:2.0:1789760738.725885:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719903996234 time 719962134256 state 0x0 ref 1 00000100:00100000:2.0:1789760738.725900:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719962394672 time 719962134256 state 0x1 ref 1 00000100:00100000:2.0:1789760738.725933:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719962394672 time 720041235621 state 0x0 ref 1 00000100:00100000:2.0:1789760738.725946:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108107480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.725994:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19324928, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.726003:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.726018:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760738.726023:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.726030:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.726050:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.779332:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19324928 -> 19329024 00010000:00010000:2.0:1789760738.779353:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779367:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779377:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779385:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.779390:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779400:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779421:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.779429:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7d3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.779530:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108107480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507234688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53582us (54258us total) trans 4294972038 rc 0/0 00000100:00100000:2.0:1789760738.779540:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4748 00000100:00100000:2.0:1789760738.779549:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719962394672 time 720041235621 state 0x2 ref 0 00000001:00080000:3.0:1789760738.780155:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972038 is committed 00000100:00100000:1.0:1789760738.783013:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.783102:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234816 00000100:00100000:2.0:1789760738.783112:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234816 00000100:00100000:2.0:1789760738.783163:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 719962394672 time 720041235621 state 0x0 ref 1 00000100:00100000:2.0:1789760738.783178:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720041496037 time 720041235621 state 0x1 ref 1 00000100:00100000:2.0:1789760738.783198:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720041496037 time 720098502524 state 0x0 ref 1 00000100:00100000:2.0:1789760738.783210:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.783257:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19329024, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.783266:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.783281:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760738.783285:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.783292:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.783306:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.819394:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972039 is committed 00002000:00010000:0.0:1789760738.819492:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19329024 -> 19333120 00010000:00010000:0.0:1789760738.819507:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819519:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819527:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819533:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760738.819538:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819547:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819566:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.819573:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7d44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.819664:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507234816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36450us (36656us total) trans 4294972039 rc 0/0 00000100:00100000:0.0:1789760738.819674:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4749 00000100:00100000:0.0:1789760738.819682:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720041496037 time 720098502524 state 0x2 ref 0 00000100:00100000:1.0:1789760738.830221:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.830333:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507234944 00000100:00100000:2.0:1789760738.830341:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507234944 00000100:00100000:2.0:1789760738.830393:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720041496037 time 720098502524 state 0x0 ref 1 00000100:00100000:2.0:1789760738.830408:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720098762940 time 720098502524 state 0x1 ref 1 00000100:00100000:2.0:1789760738.830470:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720098762940 time 720145772635 state 0x0 ref 1 00000100:00100000:2.0:1789760738.830483:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507234944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.830533:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19333120, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.830542:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.830557:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760738.830562:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.830568:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.830583:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.854145:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972040 is committed 00002000:00010000:2.0:1789760738.854228:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19333120 -> 19337216 00010000:00010000:2.0:1789760738.854242:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854261:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854271:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854279:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760738.854283:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854292:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854311:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.854329:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7d4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.854870:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507234944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24383us (24651us total) trans 4294972040 rc 0/0 00000100:00100000:2.0:1789760738.854881:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4750 00000100:00100000:2.0:1789760738.854891:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720098762940 time 720145772635 state 0x2 ref 0 00000100:00100000:2.0:1789760738.870955:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760738.871079:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235072 00000100:00100000:0.0:1789760738.871090:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235072 00000100:00100000:0.0:1789760738.871144:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720098762940 time 720145772635 state 0x0 ref 1 00000100:00100000:0.0:1789760738.871159:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720146033051 time 720145772635 state 0x1 ref 1 00000100:00100000:0.0:1789760738.871196:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720146033051 time 720186498967 state 0x0 ref 1 00000100:00100000:0.0:1789760738.871209:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507235072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760738.871257:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19337216, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:0.0:1789760738.871266:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.871282:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760738.871286:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.871292:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.871305:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.924305:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972041 is committed 00002000:00010000:3.0:1789760738.925459:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19337216 -> 19341312 00010000:00010000:3.0:1789760738.925477:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925491:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925500:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925573:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760738.925579:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925588:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925607:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.925615:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7d52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.925902:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507235072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54691us (54953us total) trans 4294972041 rc 0/0 00000100:00100000:3.0:1789760738.925914:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4751 00000100:00100000:3.0:1789760738.925924:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720146033051 time 720186498967 state 0x2 ref 0 00000100:00100000:1.0:1789760738.929624:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760738.933359:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235200 00000100:00100000:0.0:1789760738.933377:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235200 00000100:00100000:1.0:1789760738.933507:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720146033051 time 720186498967 state 0x0 ref 1 00000100:00100000:1.0:1789760738.933524:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720186759383 time 720186498967 state 0x1 ref 1 00000100:00100000:1.0:1789760738.933557:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720186759383 time 720248860502 state 0x0 ref 1 00000100:00100000:1.0:1789760738.933571:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507235200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760738.933620:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19341312, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:1.0:1789760738.933629:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.933644:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760738.933648:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.933655:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.933671:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.989215:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972042 is committed 00002000:00010000:3.0:1789760738.989222:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19341312 -> 19345408 00010000:00010000:3.0:1789760738.989238:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989252:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989262:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989269:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760738.989273:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989282:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989301:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.989315:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7d59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.989409:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507235200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55836us (59789us total) trans 4294972042 rc 0/0 00000100:00100000:3.0:1789760738.989419:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4752 00000100:00100000:3.0:1789760738.989429:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720186759383 time 720248860502 state 0x2 ref 0 00000100:00100000:1.0:1789760738.998174:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760738.998290:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235328 00000100:00100000:2.0:1789760738.998301:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235328 00000100:00100000:2.0:1789760738.998359:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720186759383 time 720248860502 state 0x0 ref 1 00000100:00100000:2.0:1789760738.998374:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720249120918 time 720248860502 state 0x1 ref 1 00000100:00100000:2.0:1789760738.998407:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720249120918 time 720313710054 state 0x0 ref 1 00000100:00100000:2.0:1789760738.998419:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507235328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760738.998466:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19345408, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0x9807 00010000:00010000:2.0:1789760738.998475:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.998491:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760738.998495:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.998501:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.998514:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.045473:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972043 is committed 00002000:00010000:2.0:1789760739.046739:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19345408 -> 19349504 00002000:00010000:2.0:1789760739.046745:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:2.0:1789760739.046748:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:2.0:1789760739.046762:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046781:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046790:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046796:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760739.046802:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046821:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046840:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.046848:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7d60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.047008:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507235328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48585us (48838us total) trans 4294972043 rc 0/0 00000100:00100000:2.0:1789760739.047021:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4753 00000100:00100000:2.0:1789760739.047031:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720249120918 time 720313710054 state 0x2 ref 0 00000100:00100000:2.0:1789760739.059580:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.059775:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235456 00000100:00100000:3.0:1789760739.059785:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235456 00000100:00100000:3.0:1789760739.059884:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720249120918 time 720313710054 state 0x0 ref 1 00000100:00100000:3.0:1789760739.059900:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720313970470 time 720313710054 state 0x1 ref 1 00000100:00100000:3.0:1789760739.059931:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720313970470 time 720375234645 state 0x0 ref 1 00000100:00100000:3.0:1789760739.059944:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.059990:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19349504, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.060000:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.060017:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.060022:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.060029:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.060062:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760739.060064:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760739.083951:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972044 is committed 00002000:00010000:3.0:1789760739.084027:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19349504 -> 19353600 00010000:00010000:3.0:1789760739.084043:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084055:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084064:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084072:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.084077:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084085:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084274:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.084283:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.084414:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24467us (24838us total) trans 4294972044 rc 0/0 00000100:00100000:3.0:1789760739.084424:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4754 00000100:00100000:3.0:1789760739.084434:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720313970470 time 720375234645 state 0x2 ref 0 00000100:00100000:0.0:1789760739.088455:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.088733:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235584 00000100:00100000:3.0:1789760739.088743:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235584 00000100:00100000:3.0:1789760739.088792:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720313970470 time 720375234645 state 0x0 ref 1 00000100:00100000:3.0:1789760739.088821:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720375495061 time 720375234645 state 0x1 ref 1 00000100:00100000:3.0:1789760739.088846:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720375495061 time 720404149400 state 0x0 ref 1 00000100:00100000:3.0:1789760739.088857:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507235584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.088902:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19353600, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.088910:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088926:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.088931:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.088938:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088951:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.121077:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972045 is committed 00002000:00010000:3.0:1789760739.121328:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19353600 -> 19357696 00010000:00010000:3.0:1789760739.121344:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121358:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121369:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121376:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.121381:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121391:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121411:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.121419:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.121514:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507235584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32654us (33063us total) trans 4294972045 rc 0/0 00000100:00100000:3.0:1789760739.121528:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4755 00000100:00100000:3.0:1789760739.121537:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720375495061 time 720404149400 state 0x2 ref 0 00000100:00100000:1.0:1789760739.125328:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760739.125426:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235712 00000100:00100000:2.0:1789760739.125436:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235712 00000100:00100000:2.0:1789760739.125492:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720375495061 time 720404149400 state 0x0 ref 1 00000100:00100000:2.0:1789760739.125507:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720404409816 time 720404149400 state 0x1 ref 1 00000100:00100000:2.0:1789760739.125570:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720404409816 time 720440872234 state 0x0 ref 1 00000100:00100000:2.0:1789760739.125583:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760739.125629:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19357696, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:2.0:1789760739.125637:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.125651:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760739.125656:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.125661:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.125674:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.150901:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972046 is committed 00002000:00010000:2.0:1789760739.150958:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19357696 -> 19361792 00010000:00010000:2.0:1789760739.150973:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.150985:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.150994:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.151001:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760739.151006:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.151015:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.151033:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.151040:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.151124:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25540us (25863us total) trans 4294972046 rc 0/0 00000100:00100000:2.0:1789760739.151134:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4756 00000100:00100000:2.0:1789760739.151143:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720404409816 time 720440872234 state 0x2 ref 0 00000100:00100000:2.0:1789760739.154857:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.155001:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235840 00000100:00100000:3.0:1789760739.155011:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235840 00000100:00100000:3.0:1789760739.155065:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720404409816 time 720440872234 state 0x0 ref 1 00000100:00100000:3.0:1789760739.155079:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720441132650 time 720440872234 state 0x1 ref 1 00000100:00100000:3.0:1789760739.155110:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720441132650 time 720470413072 state 0x0 ref 1 00000100:00100000:3.0:1789760739.155120:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.155164:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19361792, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.155173:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.155187:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.155191:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.155196:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.155210:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.180069:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972047 is committed 00002000:00010000:0.0:1789760739.180173:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19361792 -> 19365888 00010000:00010000:0.0:1789760739.180188:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180202:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180212:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180219:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760739.180224:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180233:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180252:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.180261:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.180397:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507235840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25271us (25542us total) trans 4294972047 rc 0/0 00000100:00100000:0.0:1789760739.180408:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4757 00000100:00100000:0.0:1789760739.180416:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720441132650 time 720470413072 state 0x2 ref 0 00000100:00100000:2.0:1789760739.185769:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.185864:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507235968 00000100:00100000:3.0:1789760739.185873:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507235968 00000100:00100000:3.0:1789760739.185922:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720441132650 time 720470413072 state 0x0 ref 1 00000100:00100000:3.0:1789760739.185936:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720470673488 time 720470413072 state 0x1 ref 1 00000100:00100000:3.0:1789760739.185954:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720470673488 time 720501258278 state 0x0 ref 1 00000100:00100000:3.0:1789760739.185965:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507235968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.186009:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19365888, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.186017:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.186032:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.186036:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.186044:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.186058:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.214758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972048 is committed 00002000:00010000:3.0:1789760739.214987:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19365888 -> 19369984 00010000:00010000:3.0:1789760739.215002:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215015:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215025:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215034:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.215039:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215049:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215074:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215082:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.217072:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507235968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31093us (31298us total) trans 4294972048 rc 0/0 00000100:00100000:3.0:1789760739.217087:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4758 00000100:00100000:3.0:1789760739.217097:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720470673488 time 720501258278 state 0x2 ref 0 00000100:00100000:1.0:1789760739.225090:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760739.225188:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236096 00000100:00100000:2.0:1789760739.225198:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236096 00000100:00100000:2.0:1789760739.225250:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720470673488 time 720501258278 state 0x0 ref 1 00000100:00100000:2.0:1789760739.225276:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720501518694 time 720501258278 state 0x1 ref 1 00000100:00100000:2.0:1789760739.225305:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720501518694 time 720540608911 state 0x0 ref 1 00000100:00100000:2.0:1789760739.225319:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507236096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760739.225370:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19369984, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:2.0:1789760739.225379:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.225395:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760739.225400:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.225407:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.225422:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.229475:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760739.301195:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972049 is committed 00002000:00010000:1.0:1789760739.305421:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19369984 -> 19374080 00010000:00010000:1.0:1789760739.305441:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305453:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305463:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305470:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760739.305474:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305483:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305508:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.305516:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b7d8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.305620:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507236096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 80299us (80535us total) trans 4294972049 rc 0/0 00000100:00100000:1.0:1789760739.305630:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4759 00000100:00100000:1.0:1789760739.305640:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720501518694 time 720540608911 state 0x2 ref 0 00000100:00100000:0.0:1789760739.310821:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.311118:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236224 00000100:00100000:3.0:1789760739.311130:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236224 00000100:00100000:3.0:1789760739.311182:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720501518694 time 720540608911 state 0x0 ref 1 00000100:00100000:3.0:1789760739.311201:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720540869327 time 720540608911 state 0x1 ref 1 00000100:00100000:3.0:1789760739.311289:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720540869327 time 720626590972 state 0x0 ref 1 00000100:00100000:3.0:1789760739.311305:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.311362:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19374080, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.311371:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.311386:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.311390:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.311396:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.311410:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.350555:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972050 is committed 00002000:00010000:2.0:1789760739.350738:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19374080 -> 19378176 00010000:00010000:2.0:1789760739.350753:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350765:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350779:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350786:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760739.350791:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350801:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350830:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.350837:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7d91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.350941:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39634us (40135us total) trans 4294972050 rc 0/0 00000100:00100000:2.0:1789760739.350950:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4760 00000100:00100000:2.0:1789760739.350960:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720540869327 time 720626590972 state 0x2 ref 0 00000100:00100000:2.0:1789760739.356596:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.356670:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236352 00000100:00100000:3.0:1789760739.356680:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236352 00000100:00100000:3.0:1789760739.356769:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720540869327 time 720626590972 state 0x0 ref 1 00000100:00100000:3.0:1789760739.356785:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720626851388 time 720626590972 state 0x1 ref 1 00000100:00100000:3.0:1789760739.356820:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720626851388 time 720672123471 state 0x0 ref 1 00000100:00100000:3.0:1789760739.356834:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507236352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.356880:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19378176, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.356890:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.356905:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.356909:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.356916:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.356930:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.406133:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19378176 -> 19382272 00010000:00010000:3.0:1789760739.406154:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406168:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406177:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406184:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.406189:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406198:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406217:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.406226:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7d98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.406595:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507236352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49758us (50004us total) trans 4294972051 rc 0/0 00000100:00100000:3.0:1789760739.406607:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4761 00000100:00100000:3.0:1789760739.406616:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720626851388 time 720672123471 state 0x2 ref 0 00000001:00080000:3.0:1789760739.406655:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972051 is committed 00000100:00100000:3.0:1789760739.411379:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760739.411462:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236480 00000100:00100000:1.0:1789760739.411472:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236480 00000100:00100000:1.0:1789760739.411526:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720626851388 time 720672123471 state 0x0 ref 1 00000100:00100000:1.0:1789760739.411542:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720672383887 time 720672123471 state 0x1 ref 1 00000100:00100000:1.0:1789760739.411571:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720672383887 time 720726874546 state 0x0 ref 1 00000100:00100000:1.0:1789760739.411581:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760739.411624:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19382272, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:1.0:1789760739.411633:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.411648:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760739.411653:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.411659:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.411674:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.438799:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972052 is committed 00002000:00010000:1.0:1789760739.438939:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19382272 -> 19386368 00010000:00010000:1.0:1789760739.438955:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.438967:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.438975:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.438982:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760739.438987:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.438997:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.439015:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.439028:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7d9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.439113:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27528us (27739us total) trans 4294972052 rc 0/0 00000100:00100000:1.0:1789760739.439122:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4762 00000100:00100000:1.0:1789760739.439132:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720672383887 time 720726874546 state 0x2 ref 0 00000100:00100000:2.0:1789760739.445740:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760739.445889:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236608 00000100:00100000:0.0:1789760739.445899:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236608 00000100:00100000:0.0:1789760739.445956:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720672383887 time 720726874546 state 0x0 ref 1 00000100:00100000:0.0:1789760739.445973:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720727134962 time 720726874546 state 0x1 ref 1 00000100:00100000:0.0:1789760739.446003:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720727134962 time 720761306558 state 0x0 ref 1 00000100:00100000:0.0:1789760739.446016:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760739.446062:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19386368, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:0.0:1789760739.446071:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.446086:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760739.446089:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.446096:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.446109:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.481122:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972053 is committed 00002000:00010000:3.0:1789760739.481438:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19386368 -> 19390464 00010000:00010000:3.0:1789760739.481453:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481467:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481476:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481484:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.481488:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481498:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481516:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.481525:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7da6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.481616:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35598us (35882us total) trans 4294972053 rc 0/0 00000100:00100000:3.0:1789760739.481627:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4763 00000100:00100000:3.0:1789760739.481637:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720727134962 time 720761306558 state 0x2 ref 0 00000100:00100000:1.0:1789760739.485124:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.485243:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236736 00000100:00100000:3.0:1789760739.485253:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236736 00000100:00100000:3.0:1789760739.485306:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720727134962 time 720761306558 state 0x0 ref 1 00000100:00100000:3.0:1789760739.485320:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720761566974 time 720761306558 state 0x1 ref 1 00000100:00100000:3.0:1789760739.485414:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720761566974 time 720800716242 state 0x0 ref 1 00000100:00100000:3.0:1789760739.485427:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.485478:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19390464, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.485487:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.485502:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.485507:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.485514:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.485527:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.522211:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19390464 -> 19394560 00010000:00010000:2.0:1789760739.522231:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522246:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522256:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522263:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760739.522268:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522277:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522294:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.522301:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7dad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.522392:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507236736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36963us (37275us total) trans 4294972054 rc 0/0 00000100:00100000:2.0:1789760739.522401:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4764 00000100:00100000:2.0:1789760739.522410:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720761566974 time 720800716242 state 0x2 ref 0 00000001:00080000:3.0:1789760739.524415:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972054 is committed 00000100:00100000:2.0:1789760739.529252:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760739.529433:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236864 00000100:00100000:1.0:1789760739.529443:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236864 00000100:00100000:1.0:1789760739.529496:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720761566974 time 720800716242 state 0x0 ref 1 00000100:00100000:1.0:1789760739.529512:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720800976658 time 720800716242 state 0x1 ref 1 00000100:00100000:1.0:1789760739.529554:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720800976658 time 720844856520 state 0x0 ref 1 00000100:00100000:1.0:1789760739.529567:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507236864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760739.529614:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19394560, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:1.0:1789760739.529624:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.529639:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760739.529644:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.529650:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.529664:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.548678:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760739.548720:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538273280:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760739.548918:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760739.548993:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273280 00000100:00100000:1.0:1789760739.549002:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273280 00000100:00100000:1.0:1789760739.549061:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789760739.549072:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538273280:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760739.549096:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3838836736 avail 3818913792 00000100:00100000:1.0:1789760739.549153:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538273280:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760739.549161:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1789760739.549266:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538273280:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760739.549303:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538273408:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760739.549335:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760739.549876:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273408 00000100:00100000:3.0:1789760739.549882:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273408 00000100:00100000:3.0:1789760739.549923:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:3.0:1789760739.549931:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538273408:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760739.549944:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3838836736 avail 3818913792 00000100:00100000:3.0:1789760739.549993:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538273408:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760739.549999:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1789760739.550729:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538273408:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760739.569025:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972055 is committed 00002000:00010000:3.0:1789760739.569182:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19394560 -> 19398656 00010000:00010000:3.0:1789760739.569198:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569211:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569221:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569228:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.569233:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569242:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569380:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.569389:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b7db4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.569481:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507236864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39911us (40233us total) trans 4294972055 rc 0/0 00000100:00100000:3.0:1789760739.569491:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4765 00000100:00100000:3.0:1789760739.569501:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720800976658 time 720844856520 state 0x2 ref 0 00000100:00100000:1.0:1789760739.575131:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.575325:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507236992 00000100:00100000:3.0:1789760739.575335:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507236992 00000100:00100000:3.0:1789760739.575387:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720800976658 time 720844856520 state 0x0 ref 1 00000100:00100000:3.0:1789760739.575402:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720845116936 time 720844856520 state 0x1 ref 1 00000100:00100000:3.0:1789760739.575477:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720845116936 time 720890779830 state 0x0 ref 1 00000100:00100000:3.0:1789760739.575490:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.575539:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19398656, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.575549:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.575565:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.575569:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.575577:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.575591:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.605252:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972056 is committed 00002000:00010000:1.0:1789760739.605357:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19398656 -> 19402752 00010000:00010000:1.0:1789760739.605372:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605384:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605392:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605399:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760739.605404:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605414:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605432:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.605440:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7dbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.605536:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507236992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30044us (30410us total) trans 4294972056 rc 0/0 00000100:00100000:1.0:1789760739.605546:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4766 00000100:00100000:1.0:1789760739.605556:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720845116936 time 720890779830 state 0x2 ref 0 00000100:00100000:1.0:1789760739.609929:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.610048:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237120 00000100:00100000:3.0:1789760739.610058:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237120 00000100:00100000:3.0:1789760739.610109:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720845116936 time 720890779830 state 0x0 ref 1 00000100:00100000:3.0:1789760739.610123:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720891040246 time 720890779830 state 0x1 ref 1 00000100:00100000:3.0:1789760739.610143:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720891040246 time 720925446984 state 0x0 ref 1 00000100:00100000:3.0:1789760739.610153:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.610197:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19402752, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.610206:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.610220:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760739.610225:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.610232:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.610245:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.639180:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972057 is committed 00002000:00010000:3.0:1789760739.639241:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19402752 -> 19406848 00010000:00010000:3.0:1789760739.639383:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639398:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639406:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639414:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.639419:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639428:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639448:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.639456:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.639551:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29392us (29624us total) trans 4294972057 rc 0/0 00000100:00100000:3.0:1789760739.639561:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4767 00000100:00100000:3.0:1789760739.639571:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720891040246 time 720925446984 state 0x2 ref 0 00000100:00100000:3.0:1789760739.651272:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760739.651367:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237248 00000100:00100000:1.0:1789760739.651377:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237248 00000100:00100000:1.0:1789760739.651426:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720891040246 time 720925446984 state 0x0 ref 1 00000100:00100000:1.0:1789760739.651441:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720925707400 time 720925446984 state 0x1 ref 1 00000100:00100000:1.0:1789760739.651468:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720925707400 time 720966771672 state 0x0 ref 1 00000100:00100000:1.0:1789760739.651481:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507237248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760739.651528:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19406848, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:1.0:1789760739.651538:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.651553:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760739.651559:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.651565:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.651580:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.702224:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19406848 -> 19410944 00010000:00010000:1.0:1789760739.702244:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.702416:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972058 is committed 00010000:00010000:1.0:1789760739.704124:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.704132:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.704139:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760739.704143:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.704153:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.704181:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.704189:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b7dc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.705087:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507237248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53601us (53816us total) trans 4294972058 rc 0/0 00000100:00100000:1.0:1789760739.705099:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4768 00000100:00100000:1.0:1789760739.705108:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720925707400 time 720966771672 state 0x2 ref 0 00000100:00100000:1.0:1789760739.717683:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.721598:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237376 00000100:00100000:3.0:1789760739.721615:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237376 00000100:00100000:3.0:1789760739.721690:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720925707400 time 720966771672 state 0x0 ref 1 00000100:00100000:3.0:1789760739.721726:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720967032088 time 720966771672 state 0x1 ref 1 00000100:00100000:3.0:1789760739.721751:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720967032088 time 721037054120 state 0x0 ref 1 00000100:00100000:3.0:1789760739.721767:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.721831:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19410944, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.721841:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.721857:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760739.721861:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.721867:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.721884:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.768779:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972059 is committed 00002000:00010000:3.0:1789760739.769023:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19410944 -> 19415040 00010000:00010000:3.0:1789760739.769038:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769048:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769057:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769063:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.769067:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769074:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769092:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.769099:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.769201:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47425us (51514us total) trans 4294972059 rc 0/0 00000100:00100000:3.0:1789760739.769211:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4769 00000100:00100000:3.0:1789760739.769220:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720967032088 time 721037054120 state 0x2 ref 0 00000100:00100000:2.0:1789760739.776899:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.777027:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237504 00000100:00100000:3.0:1789760739.777037:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237504 00000100:00100000:3.0:1789760739.777092:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 720967032088 time 721037054120 state 0x0 ref 1 00000100:00100000:3.0:1789760739.777106:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721037314536 time 721037054120 state 0x1 ref 1 00000100:00100000:3.0:1789760739.777138:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721037314536 time 721092441216 state 0x0 ref 1 00000100:00100000:3.0:1789760739.777150:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507237504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.777198:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19415040, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.777206:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.777221:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760739.777226:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.777231:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.777245:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.806575:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19415040 -> 19419136 00010000:00010000:3.0:1789760739.806596:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806609:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806619:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806627:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.806632:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806641:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806662:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.806670:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7dd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.806791:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507237504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29638us (29897us total) trans 4294972060 rc 0/0 00000100:00100000:3.0:1789760739.806815:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4770 00000100:00100000:3.0:1789760739.806825:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721037314536 time 721092441216 state 0x2 ref 0 00000001:00080000:0.0:1789760739.806944:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972060 is committed 00000100:00100000:1.0:1789760739.811034:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.811112:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237632 00000100:00100000:3.0:1789760739.811122:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237632 00000100:00100000:3.0:1789760739.811175:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721037314536 time 721092441216 state 0x0 ref 1 00000100:00100000:3.0:1789760739.811190:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721092701632 time 721092441216 state 0x1 ref 1 00000100:00100000:3.0:1789760739.811268:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721092701632 time 721126570712 state 0x0 ref 1 00000100:00100000:3.0:1789760739.811280:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.811331:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19419136, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.811340:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.811356:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760739.811360:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.811366:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.811380:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.850633:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972061 is committed 00002000:00010000:0.0:1789760739.850727:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19419136 -> 19423232 00010000:00010000:0.0:1789760739.850741:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850754:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850764:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850772:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760739.850777:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850786:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850819:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.850827:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7dde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.850959:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39676us (39930us total) trans 4294972061 rc 0/0 00000100:00100000:0.0:1789760739.850969:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4771 00000100:00100000:0.0:1789760739.850979:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721092701632 time 721126570712 state 0x2 ref 0 00000100:00100000:1.0:1789760739.859867:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.859941:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237760 00000100:00100000:3.0:1789760739.859953:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237760 00000100:00100000:3.0:1789760739.860116:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721092701632 time 721126570712 state 0x0 ref 1 00000100:00100000:3.0:1789760739.860132:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721126831128 time 721126570712 state 0x1 ref 1 00000100:00100000:3.0:1789760739.860231:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721126831128 time 721175532390 state 0x0 ref 1 00000100:00100000:3.0:1789760739.860245:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507237760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.860368:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19423232, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.860377:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.860392:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760739.860397:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.860403:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.860418:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.918013:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972062 is committed 00002000:00010000:1.0:1789760739.918115:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19423232 -> 19427328 00010000:00010000:1.0:1789760739.918131:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918143:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918153:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918161:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760739.918166:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918175:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918193:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.918201:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7de5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.918293:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507237760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58046us (58430us total) trans 4294972062 rc 0/0 00000100:00100000:1.0:1789760739.918302:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4772 00000100:00100000:1.0:1789760739.918312:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721126831128 time 721175532390 state 0x2 ref 0 00000100:00100000:2.0:1789760739.922152:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760739.922241:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507237888 00000100:00100000:3.0:1789760739.922260:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507237888 00000100:00100000:3.0:1789760739.922314:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721126831128 time 721175532390 state 0x0 ref 1 00000100:00100000:3.0:1789760739.922331:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721175792806 time 721175532390 state 0x1 ref 1 00000100:00100000:3.0:1789760739.922419:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721175792806 time 721237720585 state 0x0 ref 1 00000100:00100000:3.0:1789760739.922433:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760739.922482:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19427328, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:3.0:1789760739.922491:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.922507:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760739.922512:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.922519:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.922533:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.960437:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972063 is committed 00002000:00010000:3.0:1789760739.960549:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19427328 -> 19431424 00010000:00010000:3.0:1789760739.960565:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960579:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960590:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960598:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760739.960604:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960614:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960635:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.960643:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7dec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.960776:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507237888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38340us (38627us total) trans 4294972063 rc 0/0 00000100:00100000:3.0:1789760739.960787:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4773 00000100:00100000:3.0:1789760739.960796:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721175792806 time 721237720585 state 0x2 ref 0 00000100:00100000:2.0:1789760739.964685:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760739.966253:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238016 00000100:00100000:0.0:1789760739.966267:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238016 00000100:00100000:0.0:1789760739.966326:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721175792806 time 721237720585 state 0x0 ref 1 00000100:00100000:0.0:1789760739.966342:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721237981001 time 721237720585 state 0x1 ref 1 00000100:00100000:0.0:1789760739.966376:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721237981001 time 721281679082 state 0x0 ref 1 00000100:00100000:0.0:1789760739.966389:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507238016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760739.966438:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19431424, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:0.0:1789760739.966447:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.966467:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760739.966471:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.966478:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.966492:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.992949:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19431424 -> 19435520 00010000:00010000:0.0:1789760739.992968:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.992980:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.992989:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.992997:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760739.993002:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.993010:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.993027:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.993036:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7df3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.993124:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507238016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26732us (28444us total) trans 4294972064 rc 0/0 00000100:00100000:0.0:1789760739.993134:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4774 00000100:00100000:0.0:1789760739.993143:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721237981001 time 721281679082 state 0x2 ref 0 00000001:00080000:0.0:1789760739.993248:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972064 is committed 00000100:00100000:0.0:1789760739.996431:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760740.001336:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238144 00000100:00100000:1.0:1789760740.001353:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238144 00000100:00100000:1.0:1789760740.001417:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721237981001 time 721281679082 state 0x0 ref 1 00000100:00100000:1.0:1789760740.001435:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721281939498 time 721281679082 state 0x1 ref 1 00000100:00100000:1.0:1789760740.001456:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721281939498 time 721316759942 state 0x0 ref 1 00000100:00100000:1.0:1789760740.001468:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507238144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760740.001514:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19435520, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0x9807 00010000:00010000:1.0:1789760740.001523:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.001538:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760740.001542:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.001548:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.001562:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.028627:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19435520 -> 19439616 00002000:00010000:2.0:1789760740.028636:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:2.0:1789760740.028638:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:2.0:1789760740.028651:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028662:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028671:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028679:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.028683:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028718:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028745:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.028753:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7dfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.028860:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507238144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27388us (32434us total) trans 4294972065 rc 0/0 00000100:00100000:2.0:1789760740.028871:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4775 00000100:00100000:2.0:1789760740.028882:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721281939498 time 721316759942 state 0x2 ref 0 00000001:00080000:3.0:1789760740.029419:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972065 is committed 00000100:00100000:2.0:1789760740.032760:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.032882:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238272 00000100:00100000:3.0:1789760740.032892:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238272 00000100:00100000:3.0:1789760740.032942:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721281939498 time 721316759942 state 0x0 ref 1 00000100:00100000:3.0:1789760740.032957:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721317020358 time 721316759942 state 0x1 ref 1 00000100:00100000:3.0:1789760740.032989:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721317020358 time 721348292783 state 0x0 ref 1 00000100:00100000:3.0:1789760740.033002:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507238272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.033044:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19439616, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.033053:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.033072:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760740.033076:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.033082:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.033096:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.066067:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972066 is committed 00000100:00100000:1.0:1789760740.066403:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760740.066420:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538273536:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760740.066483:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760740.066547:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273536 00000100:00100000:1.0:1789760740.066555:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273536 00000100:00100000:1.0:1789760740.066596:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:1.0:1789760740.066605:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538273536:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760740.066640:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760740.066724:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538273536:12345-0@lo:41:osp-pre-1-0.0 Request processed in 120us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760740.066732:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:1.0:1789760740.066780:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538273536:0@lo:41:osp-pre-1-0.0 00002000:00010000:0.0:1789760740.067060:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19439616 -> 19443712 00010000:00010000:0.0:1789760740.067074:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067086:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067096:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067103:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760740.067108:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067117:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067137:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.067216:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.067288:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507238272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34285us (34535us total) trans 4294972066 rc 0/0 00000100:00100000:0.0:1789760740.067297:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4776 00000100:00100000:0.0:1789760740.067305:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721317020358 time 721348292783 state 0x2 ref 0 00000100:00100000:1.0:1789760740.075513:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760740.075617:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238400 00000100:00100000:0.0:1789760740.075626:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238400 00000100:00100000:0.0:1789760740.075677:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721317020358 time 721348292783 state 0x0 ref 1 00000100:00100000:0.0:1789760740.075880:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721348553199 time 721348292783 state 0x1 ref 1 00000100:00100000:1.0:1789760740.076017:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721348553199 time 721391306140 state 0x0 ref 1 00000100:00100000:1.0:1789760740.076032:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507238400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760740.076087:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19443712, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:1.0:1789760740.076096:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.076112:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760740.076116:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.076124:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.076138:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.117512:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972067 is committed 00002000:00010000:2.0:1789760740.117664:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19443712 -> 19447808 00010000:00010000:2.0:1789760740.117680:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117756:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117768:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117776:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.117780:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117790:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117820:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.117829:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7e08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.118026:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507238400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41992us (42516us total) trans 4294972067 rc 0/0 00000100:00100000:2.0:1789760740.118036:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4777 00000100:00100000:2.0:1789760740.118045:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721348553199 time 721391306140 state 0x2 ref 0 00000100:00100000:1.0:1789760740.125777:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.129110:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238528 00000100:00100000:2.0:1789760740.129125:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238528 00000100:00100000:2.0:1789760740.129248:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721348553199 time 721391306140 state 0x0 ref 1 00000100:00100000:2.0:1789760740.129263:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721391566556 time 721391306140 state 0x1 ref 1 00000100:00100000:2.0:1789760740.129337:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721391566556 time 721444639150 state 0x0 ref 1 00000100:00100000:2.0:1789760740.129351:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507238528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.129401:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19447808, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.129410:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.129425:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760740.129430:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.129436:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.129450:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.172661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972068 is committed 00002000:00010000:0.0:1789760740.172992:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19447808 -> 19451904 00010000:00010000:0.0:1789760740.173007:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173020:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173028:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173041:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760740.173045:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173055:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173074:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.173082:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.173330:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507238528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43976us (47557us total) trans 4294972068 rc 0/0 00000100:00100000:0.0:1789760740.173341:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4778 00000100:00100000:0.0:1789760740.173351:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721391566556 time 721444639150 state 0x2 ref 0 00000100:00100000:1.0:1789760740.177480:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.177559:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238656 00000100:00100000:3.0:1789760740.177569:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238656 00000100:00100000:3.0:1789760740.177623:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721391566556 time 721444639150 state 0x0 ref 1 00000100:00100000:3.0:1789760740.177638:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721444899566 time 721444639150 state 0x1 ref 1 00000100:00100000:3.0:1789760740.177672:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721444899566 time 721492975303 state 0x0 ref 1 00000100:00100000:3.0:1789760740.177686:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507238656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.177938:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19451904, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.177947:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.177964:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760740.177968:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.177975:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.177990:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.241526:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19451904 -> 19456000 00010000:00010000:2.0:1789760740.241544:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241556:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241565:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241574:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.241579:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241587:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241607:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.241615:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.241738:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507238656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64049us (64264us total) trans 4294972069 rc 0/0 00000100:00100000:2.0:1789760740.241748:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4779 00000100:00100000:2.0:1789760740.241757:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721444899566 time 721492975303 state 0x2 ref 0 00000001:00080000:3.0:1789760740.241869:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972069 is committed 00000100:00100000:1.0:1789760740.246649:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.246774:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238784 00000100:00100000:2.0:1789760740.246783:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238784 00000100:00100000:2.0:1789760740.246858:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721444899566 time 721492975303 state 0x0 ref 1 00000100:00100000:2.0:1789760740.246873:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721493235719 time 721492975303 state 0x1 ref 1 00000100:00100000:2.0:1789760740.246901:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721493235719 time 721562204830 state 0x0 ref 1 00000100:00100000:2.0:1789760740.246914:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507238784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.246964:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19456000, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.246972:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.246988:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760740.246992:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.246998:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.247018:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.253479:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760740.277805:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972070 is committed 00002000:00010000:3.0:1789760740.278028:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19456000 -> 19460096 00010000:00010000:3.0:1789760740.278043:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278055:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278064:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278072:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.278077:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278086:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278105:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.278113:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7e1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.278361:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507238784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31444us (31716us total) trans 4294972070 rc 0/0 00000100:00100000:3.0:1789760740.278372:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4780 00000100:00100000:3.0:1789760740.278381:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721493235719 time 721562204830 state 0x2 ref 0 00000100:00100000:1.0:1789760740.282229:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760740.282375:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507238912 00000100:00100000:0.0:1789760740.282385:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507238912 00000100:00100000:0.0:1789760740.282438:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721493235719 time 721562204830 state 0x0 ref 1 00000100:00100000:0.0:1789760740.282454:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721562465246 time 721562204830 state 0x1 ref 1 00000100:00100000:0.0:1789760740.282518:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721562465246 time 721597820487 state 0x0 ref 1 00000100:00100000:0.0:1789760740.282535:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507238912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760740.282586:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19460096, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:0.0:1789760740.282595:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.282611:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760740.282615:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.282622:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.282635:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.367285:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972071 is committed 00002000:00010000:3.0:1789760740.367850:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19460096 -> 19464192 00010000:00010000:3.0:1789760740.367867:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367881:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367889:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367896:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.367901:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367909:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367927:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.367940:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7e24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.368040:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507238912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 85504us (85816us total) trans 4294972071 rc 0/0 00000100:00100000:3.0:1789760740.368049:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4781 00000100:00100000:3.0:1789760740.368058:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721562465246 time 721597820487 state 0x2 ref 0 00000100:00100000:0.0:1789760740.371281:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.371352:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239040 00000100:00100000:2.0:1789760740.371362:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239040 00000100:00100000:2.0:1789760740.371414:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721562465246 time 721597820487 state 0x0 ref 1 00000100:00100000:2.0:1789760740.371429:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721598080903 time 721597820487 state 0x1 ref 1 00000100:00100000:2.0:1789760740.371525:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721598080903 time 721686826847 state 0x0 ref 1 00000100:00100000:2.0:1789760740.371539:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507239040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.371587:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19464192, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.371595:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.371612:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760740.371616:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.371623:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.371636:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.420050:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972072 is committed 00002000:00010000:2.0:1789760740.420221:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19464192 -> 19468288 00010000:00010000:2.0:1789760740.420236:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420378:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420388:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420396:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.420400:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420409:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420428:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420436:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7e2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.420568:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507239040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49026us (49290us total) trans 4294972072 rc 0/0 00000100:00100000:2.0:1789760740.420583:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4782 00000100:00100000:2.0:1789760740.420592:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721598080903 time 721686826847 state 0x2 ref 0 00000100:00100000:2.0:1789760740.430646:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.430980:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239168 00000100:00100000:3.0:1789760740.430994:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239168 00000100:00100000:3.0:1789760740.431064:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721598080903 time 721686826847 state 0x0 ref 1 00000100:00100000:3.0:1789760740.431080:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721687087263 time 721686826847 state 0x1 ref 1 00000100:00100000:3.0:1789760740.431428:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721687087263 time 721746729451 state 0x0 ref 1 00000100:00100000:3.0:1789760740.431443:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507239168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.431494:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19468288, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.431503:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.431519:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760740.431525:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.431531:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.431549:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.490622:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19468288 -> 19472384 00010000:00010000:1.0:1789760740.490645:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490659:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490669:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490682:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760740.490687:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490748:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490769:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.490777:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7e32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.490911:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507239168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59467us (60270us total) trans 4294972073 rc 0/0 00000100:00100000:1.0:1789760740.490922:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4783 00000100:00100000:1.0:1789760740.490933:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721687087263 time 721746729451 state 0x2 ref 0 00000001:00080000:2.0:1789760740.491092:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972073 is committed 00000100:00100000:0.0:1789760740.493962:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.494048:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239296 00000100:00100000:3.0:1789760740.494058:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239296 00000100:00100000:3.0:1789760740.494109:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721687087263 time 721746729451 state 0x0 ref 1 00000100:00100000:3.0:1789760740.494125:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721746989867 time 721746729451 state 0x1 ref 1 00000100:00100000:3.0:1789760740.494190:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721746989867 time 721809492537 state 0x0 ref 1 00000100:00100000:3.0:1789760740.494203:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.494256:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19472384, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.494266:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.494281:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760740.494286:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.494292:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.494307:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.526112:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972074 is committed 00002000:00010000:3.0:1789760740.526273:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19472384 -> 19476480 00010000:00010000:3.0:1789760740.526290:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526303:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526311:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526319:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.526324:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526333:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526353:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.526360:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.526467:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32260us (32509us total) trans 4294972074 rc 0/0 00000100:00100000:3.0:1789760740.526477:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4784 00000100:00100000:3.0:1789760740.526488:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721746989867 time 721809492537 state 0x2 ref 0 00000100:00100000:3.0:1789760740.530736:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.530836:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239424 00000100:00100000:2.0:1789760740.530851:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239424 00000100:00100000:2.0:1789760740.530920:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721746989867 time 721809492537 state 0x0 ref 1 00000100:00100000:2.0:1789760740.530935:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721809752953 time 721809492537 state 0x1 ref 1 00000100:00100000:2.0:1789760740.530970:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721809752953 time 721846273552 state 0x0 ref 1 00000100:00100000:2.0:1789760740.530985:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.531033:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19476480, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.531042:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531057:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760740.531062:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.531068:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531082:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.571929:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760740.575949:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234677100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538273664:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760740.576030:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760740.576139:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273664 00000100:00100000:3.0:1789760740.576148:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273664 00000100:00100000:3.0:1789760740.576207:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1789760740.576217:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108107480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538273664:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760740.576258:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760740.576309:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108107480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538273664:12345-0@lo:41:osp-pre-0-1.0 Request processed in 93us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760740.576316:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1789760740.576748:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234677100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538273664:0@lo:41:osp-pre-0-1.0 00002000:00010000:2.0:1789760740.584030:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19476480 -> 19480576 00010000:00010000:2.0:1789760740.584052:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584065:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584074:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584082:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.584086:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584108:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584128:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.584136:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.584237:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507239424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53249us (53565us total) trans 4294972075 rc 0/0 00000100:00100000:2.0:1789760740.584251:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4785 00000100:00100000:2.0:1789760740.584262:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721809752953 time 721846273552 state 0x2 ref 0 00000001:00080000:1.0:1789760740.584946:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972075 is committed 00000100:00100000:2.0:1789760740.589430:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760740.589580:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239552 00000100:00100000:1.0:1789760740.589590:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239552 00000100:00100000:1.0:1789760740.589640:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721809752953 time 721846273552 state 0x0 ref 1 00000100:00100000:1.0:1789760740.589656:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721846533968 time 721846273552 state 0x1 ref 1 00000100:00100000:1.0:1789760740.589690:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721846533968 time 721904992937 state 0x0 ref 1 00000100:00100000:1.0:1789760740.589720:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760740.589768:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19480576, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:1.0:1789760740.589778:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.589802:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760740.589807:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.589813:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.589828:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.614126:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19480576 -> 19484672 00010000:00010000:2.0:1789760740.614144:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614157:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614165:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614172:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.614177:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614197:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614219:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.614226:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7e47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.614322:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507239552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24598us (24897us total) trans 4294972076 rc 0/0 00000100:00100000:2.0:1789760740.614332:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4786 00000100:00100000:2.0:1789760740.614342:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721846533968 time 721904992937 state 0x2 ref 0 00000001:00080000:0.0:1789760740.614724:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972076 is committed 00000100:00100000:2.0:1789760740.620652:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.620824:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239680 00000100:00100000:3.0:1789760740.620903:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239680 00000100:00100000:3.0:1789760740.621053:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721846533968 time 721904992937 state 0x0 ref 1 00000100:00100000:3.0:1789760740.621072:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721905253353 time 721904992937 state 0x1 ref 1 00000100:00100000:3.0:1789760740.621111:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721905253353 time 721936413922 state 0x0 ref 1 00000100:00100000:3.0:1789760740.621127:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507239680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.621197:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19484672, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.621208:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.621234:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760740.621581:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.621591:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.621611:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.668774:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972077 is committed 00002000:00010000:2.0:1789760740.669044:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19484672 -> 19488768 00010000:00010000:2.0:1789760740.669064:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669079:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669089:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669097:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.669102:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669113:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669138:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669152:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7e4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.669288:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507239680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48159us (48645us total) trans 4294972077 rc 0/0 00000100:00100000:2.0:1789760740.670216:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4787 00000100:00100000:2.0:1789760740.670236:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721905253353 time 721936413922 state 0x2 ref 0 00000100:00100000:1.0:1789760740.672902:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.673084:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239808 00000100:00100000:2.0:1789760740.673097:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239808 00000100:00100000:2.0:1789760740.673169:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721905253353 time 721936413922 state 0x0 ref 1 00000100:00100000:2.0:1789760740.673187:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721936674338 time 721936413922 state 0x1 ref 1 00000100:00100000:2.0:1789760740.673219:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721936674338 time 721988521702 state 0x0 ref 1 00000100:00100000:2.0:1789760740.673235:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.674230:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19488768, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.674249:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.674280:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760740.674286:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.675608:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.675648:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.785937:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972078 is committed 00002000:00010000:2.0:1789760740.786099:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19488768 -> 19492864 00010000:00010000:2.0:1789760740.786113:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786126:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786135:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786142:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760740.786147:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786157:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786176:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.786183:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.786271:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507239808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 113036us (113379us total) trans 4294972078 rc 0/0 00000100:00100000:2.0:1789760740.786281:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4788 00000100:00100000:2.0:1789760740.786289:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721936674338 time 721988521702 state 0x2 ref 0 00000100:00100000:3.0:1789760740.790514:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.790589:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507239936 00000100:00100000:2.0:1789760740.790598:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507239936 00000100:00100000:2.0:1789760740.790646:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721936674338 time 721988521702 state 0x0 ref 1 00000100:00100000:2.0:1789760740.790660:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721988782118 time 721988521702 state 0x1 ref 1 00000100:00100000:2.0:1789760740.790689:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721988782118 time 722105993229 state 0x0 ref 1 00000100:00100000:3.0:1789760740.790902:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507239936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.790948:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19492864, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.790956:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.790970:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760740.790974:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.790981:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.790994:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.824636:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19492864 -> 19496960 00010000:00010000:3.0:1789760740.824657:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824677:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824687:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824716:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.824721:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824730:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824751:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.824760:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.824936:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507239936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34079us (34426us total) trans 4294972079 rc 0/0 00000100:00100000:3.0:1789760740.824947:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4789 00000100:00100000:3.0:1789760740.824956:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721988782118 time 722105993229 state 0x2 ref 0 00000001:00080000:2.0:1789760740.826731:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972079 is committed 00000100:00100000:0.0:1789760740.829306:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.829451:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240064 00000100:00100000:2.0:1789760740.829463:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240064 00000100:00100000:2.0:1789760740.829518:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 721988782118 time 722105993229 state 0x0 ref 1 00000100:00100000:2.0:1789760740.829533:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722106253645 time 722105993229 state 0x1 ref 1 00000100:00100000:2.0:1789760740.829565:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722106253645 time 722144868445 state 0x0 ref 1 00000100:00100000:2.0:1789760740.829579:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.829629:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19496960, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.829639:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.829655:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760740.829660:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.829668:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.829683:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.858910:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972080 is committed 00002000:00010000:3.0:1789760740.859011:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19496960 -> 19501056 00010000:00010000:3.0:1789760740.859027:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859040:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859049:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859056:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.859060:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859069:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859087:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859094:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b7e63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.859181:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29600us (29878us total) trans 4294972080 rc 0/0 00000100:00100000:3.0:1789760740.859190:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4790 00000100:00100000:3.0:1789760740.859200:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722106253645 time 722144868445 state 0x2 ref 0 00000100:00100000:1.0:1789760740.864892:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760740.865142:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240192 00000100:00100000:2.0:1789760740.865152:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240192 00000100:00100000:2.0:1789760740.865207:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722106253645 time 722144868445 state 0x0 ref 1 00000100:00100000:2.0:1789760740.865222:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722145128861 time 722144868445 state 0x1 ref 1 00000100:00100000:2.0:1789760740.865256:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722145128861 time 722180559517 state 0x0 ref 1 00000100:00100000:2.0:1789760740.865268:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507240192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760740.865316:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19501056, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:2.0:1789760740.865325:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.865340:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760740.865345:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.865351:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.865523:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.915088:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972081 is committed 00002000:00010000:0.0:1789760740.915240:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19501056 -> 19505152 00010000:00010000:0.0:1789760740.915257:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915268:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915283:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915290:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760740.915295:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915304:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915324:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.915389:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7e6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.915581:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507240192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50309us (50693us total) trans 4294972081 rc 0/0 00000100:00100000:0.0:1789760740.915591:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4791 00000100:00100000:0.0:1789760740.915601:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722145128861 time 722180559517 state 0x2 ref 0 00000100:00100000:2.0:1789760740.920869:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.920991:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240320 00000100:00100000:3.0:1789760740.921001:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240320 00000100:00100000:3.0:1789760740.921053:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722145128861 time 722180559517 state 0x0 ref 1 00000100:00100000:3.0:1789760740.921069:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722180819933 time 722180559517 state 0x1 ref 1 00000100:00100000:3.0:1789760740.921101:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722180819933 time 722236403870 state 0x0 ref 1 00000100:00100000:3.0:1789760740.921113:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.921166:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19505152, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.921175:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.921192:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760740.921197:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.921203:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.921218:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.954267:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972082 is committed 00002000:00010000:0.0:1789760740.954470:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19505152 -> 19509248 00010000:00010000:0.0:1789760740.954485:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954504:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954514:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954522:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760740.954527:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954535:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954553:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.954561:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7e71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.954648:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33532us (33784us total) trans 4294972082 rc 0/0 00000100:00100000:0.0:1789760740.954658:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4792 00000100:00100000:0.0:1789760740.954669:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722180819933 time 722236403870 state 0x2 ref 0 00000100:00100000:2.0:1789760740.960656:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760740.961071:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240448 00000100:00100000:3.0:1789760740.961080:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240448 00000100:00100000:3.0:1789760740.961137:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722180819933 time 722236403870 state 0x0 ref 1 00000100:00100000:3.0:1789760740.961167:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722236664286 time 722236403870 state 0x1 ref 1 00000100:00100000:3.0:1789760740.961249:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722236664286 time 722276552270 state 0x0 ref 1 00000100:00100000:3.0:1789760740.961262:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507240448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760740.961318:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19509248, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:3.0:1789760740.961333:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.961348:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760740.961353:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.961360:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.961375:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.996080:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972083 is committed 00002000:00010000:3.0:1789760740.996254:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19509248 -> 19513344 00010000:00010000:3.0:1789760740.996278:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996293:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996304:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996312:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760740.996319:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996330:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996355:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.996364:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7e78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.996514:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507240448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35244us (35859us total) trans 4294972083 rc 0/0 00000100:00100000:3.0:1789760740.996525:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4793 00000100:00100000:3.0:1789760740.996538:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722236664286 time 722276552270 state 0x2 ref 0 00000100:00100000:1.0:1789760741.004687:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760741.004850:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240576 00000100:00100000:0.0:1789760741.004865:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240576 00000100:00100000:0.0:1789760741.004968:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722236664286 time 722276552270 state 0x0 ref 1 00000100:00100000:0.0:1789760741.004987:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722276812686 time 722276552270 state 0x1 ref 1 00000100:00100000:0.0:1789760741.005023:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722276812686 time 722320326345 state 0x0 ref 1 00000100:00100000:0.0:1789760741.005039:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760741.005105:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19513344, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0x9807 00010000:00010000:0.0:1789760741.005116:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.005134:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760741.005139:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.005147:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.005165:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760741.005737:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760741.046312:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972084 is committed 00002000:00010000:3.0:1789760741.046432:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19513344 -> 19517440 00002000:00010000:3.0:1789760741.046437:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:3.0:1789760741.046441:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:3.0:1789760741.046457:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046472:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046482:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046492:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760741.046498:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046508:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046531:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.046538:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7e7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.046652:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41610us (41974us total) trans 4294972084 rc 0/0 00000100:00100000:3.0:1789760741.046663:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4794 00000100:00100000:3.0:1789760741.046676:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722276812686 time 722320326345 state 0x2 ref 0 00000100:00100000:3.0:1789760741.053947:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760741.054047:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240704 00000100:00100000:0.0:1789760741.054060:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240704 00000100:00100000:0.0:1789760741.054148:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722276812686 time 722320326345 state 0x0 ref 1 00000100:00100000:0.0:1789760741.054166:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722320586761 time 722320326345 state 0x1 ref 1 00000100:00100000:0.0:1789760741.054212:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722320586761 time 722369511831 state 0x0 ref 1 00000100:00100000:0.0:1789760741.054232:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507240704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760741.054308:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19517440, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:0.0:1789760741.054319:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.054339:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760741.054344:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.054352:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.054369:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.090091:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972085 is committed 00002000:00010000:1.0:1789760741.090095:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19517440 -> 19521536 00010000:00010000:1.0:1789760741.090117:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090133:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090149:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090156:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.090161:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090167:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090189:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.090195:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7e86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.090305:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507240704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36065us (36362us total) trans 4294972085 rc 0/0 00000100:00100000:1.0:1789760741.090317:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4795 00000100:00100000:1.0:1789760741.090329:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722320586761 time 722369511831 state 0x2 ref 0 00000100:00100000:2.0:1789760741.092960:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760741.093063:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240832 00000100:00100000:3.0:1789760741.093074:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240832 00000100:00100000:3.0:1789760741.093133:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722320586761 time 722369511831 state 0x0 ref 1 00000100:00100000:3.0:1789760741.093149:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722369772247 time 722369511831 state 0x1 ref 1 00000100:00100000:3.0:1789760741.093180:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722369772247 time 722408483571 state 0x0 ref 1 00000100:00100000:3.0:1789760741.093194:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507240832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760741.093257:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19521536, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:3.0:1789760741.093268:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.093298:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760741.093304:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.093312:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.093328:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.124912:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19521536 -> 19525632 00010000:00010000:2.0:1789760741.124940:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.124955:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.124965:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.124974:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.124980:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.124995:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.125022:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.125031:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7e8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.125156:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507240832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31956us (32204us total) trans 4294972086 rc 0/0 00000100:00100000:2.0:1789760741.125168:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4796 00000100:00100000:2.0:1789760741.125179:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722369772247 time 722408483571 state 0x2 ref 0 00000001:00080000:1.0:1789760741.125352:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972086 is committed 00000100:00100000:3.0:1789760741.129958:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760741.130084:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507240960 00000100:00100000:0.0:1789760741.130101:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507240960 00000100:00100000:0.0:1789760741.130182:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722369772247 time 722408483571 state 0x0 ref 1 00000100:00100000:0.0:1789760741.130202:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722408743987 time 722408483571 state 0x1 ref 1 00000100:00100000:0.0:1789760741.130252:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722408743987 time 722445553900 state 0x0 ref 1 00000100:00100000:0.0:1789760741.130267:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760741.130330:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19525632, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:0.0:1789760741.130352:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.130378:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760741.130384:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.130392:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.130409:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.167922:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972087 is committed 00002000:00010000:2.0:1789760741.167983:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19525632 -> 19529728 00010000:00010000:2.0:1789760741.168003:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168018:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168028:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168035:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.168042:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168053:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168080:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.168089:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7e94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.168206:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507240960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37935us (38251us total) trans 4294972087 rc 0/0 00000100:00100000:2.0:1789760741.168220:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4797 00000100:00100000:2.0:1789760741.168233:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722408743987 time 722445553900 state 0x2 ref 0 00000100:00100000:0.0:1789760741.171505:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760741.171590:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241088 00000100:00100000:1.0:1789760741.171603:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241088 00000100:00100000:1.0:1789760741.171672:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722408743987 time 722445553900 state 0x0 ref 1 00000100:00100000:1.0:1789760741.171889:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722445814316 time 722445553900 state 0x1 ref 1 00000100:00100000:1.0:1789760741.171927:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722445814316 time 722487228825 state 0x0 ref 1 00000100:00100000:1.0:1789760741.171942:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507241088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760741.172005:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19529728, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:1.0:1789760741.172129:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.172149:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760741.172155:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.172163:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.172181:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760741.205420:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19529728 -> 19533824 00010000:00010000:0.0:1789760741.205452:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205468:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205478:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205486:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760741.205491:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205502:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205534:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.205542:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b7e9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.205670:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507241088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33723us (34170us total) trans 4294972088 rc 0/0 00000100:00100000:0.0:1789760741.205681:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4798 00000100:00100000:0.0:1789760741.205743:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722445814316 time 722487228825 state 0x2 ref 0 00000001:00080000:2.0:1789760741.208115:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972088 is committed 00000100:00100000:0.0:1789760741.210430:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.210531:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241216 00000100:00100000:2.0:1789760741.210544:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241216 00000100:00100000:2.0:1789760741.210620:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722445814316 time 722487228825 state 0x0 ref 1 00000100:00100000:2.0:1789760741.210639:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722487489241 time 722487228825 state 0x1 ref 1 00000100:00100000:2.0:1789760741.210675:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722487489241 time 722525978100 state 0x0 ref 1 00000100:00100000:2.0:1789760741.210985:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507241216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.211056:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19533824, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.211067:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.211087:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.211092:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.211100:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.211118:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.247320:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972089 is committed 00002000:00010000:3.0:1789760741.247441:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19533824 -> 19537920 00010000:00010000:3.0:1789760741.247461:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247477:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247487:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247495:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760741.247500:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247511:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247538:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.247546:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b7ea2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.247667:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507241216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36972us (37284us total) trans 4294972089 rc 0/0 00000100:00100000:3.0:1789760741.247679:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4799 00000100:00100000:3.0:1789760741.247732:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722487489241 time 722525978100 state 0x2 ref 0 00000100:00100000:3.0:1789760741.260434:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.260535:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241344 00000100:00100000:2.0:1789760741.260548:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241344 00000100:00100000:2.0:1789760741.260623:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722487489241 time 722525978100 state 0x0 ref 1 00000100:00100000:2.0:1789760741.260642:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722526238516 time 722525978100 state 0x1 ref 1 00000100:00100000:2.0:1789760741.260681:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722526238516 time 722575983684 state 0x0 ref 1 00000100:00100000:2.0:1789760741.260742:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.260819:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19537920, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.260831:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.260856:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.260862:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.261494:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.261619:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.276115:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760741.295571:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972090 is committed 00002000:00010000:1.0:1789760741.295737:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19537920 -> 19542016 00010000:00010000:1.0:1789760741.295758:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.295773:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.295783:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.295864:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.295871:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.295882:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.296111:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.296121:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ea9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.296240:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35540us (35814us total) trans 4294972090 rc 0/0 00000100:00100000:1.0:1789760741.296303:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4800 00000100:00100000:1.0:1789760741.296318:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722526238516 time 722575983684 state 0x2 ref 0 00000100:00100000:0.0:1789760741.299104:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.299298:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241472 00000100:00100000:2.0:1789760741.299311:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241472 00000100:00100000:2.0:1789760741.299387:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722526238516 time 722575983684 state 0x0 ref 1 00000100:00100000:2.0:1789760741.299407:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722576244100 time 722575983684 state 0x1 ref 1 00000100:00100000:2.0:1789760741.299440:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722576244100 time 722614743101 state 0x0 ref 1 00000100:00100000:2.0:1789760741.299455:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507241472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.299521:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19542016, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.299532:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.299551:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.299557:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.299565:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.299588:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.344217:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972091 is committed 00002000:00010000:3.0:1789760741.344343:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19542016 -> 19546112 00010000:00010000:3.0:1789760741.344371:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344387:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344396:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344409:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760741.344415:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344426:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344452:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.344461:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b7eb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.344582:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507241472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45123us (45486us total) trans 4294972091 rc 0/0 00000100:00100000:3.0:1789760741.344594:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4801 00000100:00100000:3.0:1789760741.344606:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722576244100 time 722614743101 state 0x2 ref 0 00000100:00100000:0.0:1789760741.351499:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760741.351580:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241600 00000100:00100000:3.0:1789760741.351594:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241600 00000100:00100000:3.0:1789760741.351669:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722576244100 time 722614743101 state 0x0 ref 1 00000100:00100000:3.0:1789760741.351687:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722615003517 time 722614743101 state 0x1 ref 1 00000100:00100000:3.0:1789760741.351884:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722615003517 time 722667185560 state 0x0 ref 1 00000100:00100000:3.0:1789760741.351903:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760741.351979:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19546112, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:3.0:1789760741.351990:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.352009:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760741.352015:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.352023:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.352040:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.387672:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972092 is committed 00002000:00010000:2.0:1789760741.387974:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19546112 -> 19550208 00010000:00010000:2.0:1789760741.387996:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388012:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388023:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388032:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.388038:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388048:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388074:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.388083:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7eb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.388262:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36354us (36770us total) trans 4294972092 rc 0/0 00000100:00100000:2.0:1789760741.388274:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4802 00000100:00100000:2.0:1789760741.388286:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722615003517 time 722667185560 state 0x2 ref 0 00000100:00100000:1.0:1789760741.396598:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.396966:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241728 00000100:00100000:2.0:1789760741.396981:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241728 00000100:00100000:2.0:1789760741.397064:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722615003517 time 722667185560 state 0x0 ref 1 00000100:00100000:2.0:1789760741.397084:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722667445976 time 722667185560 state 0x1 ref 1 00000100:00100000:2.0:1789760741.397162:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722667445976 time 722712463450 state 0x0 ref 1 00000100:00100000:2.0:1789760741.397176:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507241728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.397255:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19550208, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.397267:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.397287:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.397292:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.397300:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.397315:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.430734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972093 is committed 00002000:00010000:2.0:1789760741.431080:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19550208 -> 19554304 00010000:00010000:2.0:1789760741.431101:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431117:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431128:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431136:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.431141:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431151:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431176:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.431184:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ebe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.431311:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507241728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34130us (34722us total) trans 4294972093 rc 0/0 00000100:00100000:2.0:1789760741.431328:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4803 00000100:00100000:2.0:1789760741.431342:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722667445976 time 722712463450 state 0x2 ref 0 00000100:00100000:0.0:1789760741.436622:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760741.436730:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241856 00000100:00100000:3.0:1789760741.436743:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241856 00000100:00100000:3.0:1789760741.436881:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722667445976 time 722712463450 state 0x0 ref 1 00000100:00100000:3.0:1789760741.436902:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722712723866 time 722712463450 state 0x1 ref 1 00000100:00100000:3.0:1789760741.436937:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722712723866 time 722752240287 state 0x0 ref 1 00000100:00100000:3.0:1789760741.436952:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760741.437017:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19554304, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:3.0:1789760741.437029:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.437055:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760741.437061:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.437068:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.437085:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.478442:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972094 is committed 00002000:00010000:2.0:1789760741.478567:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19554304 -> 19558400 00010000:00010000:2.0:1789760741.478587:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478602:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478613:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478621:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.478627:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478637:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478663:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.478671:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ec5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.479112:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507241856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42153us (42495us total) trans 4294972094 rc 0/0 00000100:00100000:2.0:1789760741.479125:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4804 00000100:00100000:2.0:1789760741.479137:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722712723866 time 722752240287 state 0x2 ref 0 00000100:00100000:0.0:1789760741.486136:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.486439:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507241984 00000100:00100000:2.0:1789760741.486453:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507241984 00000100:00100000:2.0:1789760741.486531:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722712723866 time 722752240287 state 0x0 ref 1 00000100:00100000:2.0:1789760741.486550:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722752500703 time 722752240287 state 0x1 ref 1 00000100:00100000:2.0:1789760741.486624:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722752500703 time 722801925541 state 0x0 ref 1 00000100:00100000:2.0:1789760741.486639:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507241984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.486819:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19558400, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.486841:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.486862:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.486867:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.486875:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.486892:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.519227:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19558400 -> 19562496 00010000:00010000:2.0:1789760741.519256:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519270:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519280:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519289:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.519294:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519304:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519331:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.519339:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ecc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.519497:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507241984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32854us (33368us total) trans 4294972095 rc 0/0 00000100:00100000:2.0:1789760741.519513:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4805 00000100:00100000:2.0:1789760741.519525:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722752500703 time 722801925541 state 0x2 ref 0 00000001:00080000:3.0:1789760741.519620:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972095 is committed 00000100:00100000:1.0:1789760741.524983:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.525509:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242112 00000100:00100000:2.0:1789760741.525524:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242112 00000100:00100000:2.0:1789760741.525601:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722752500703 time 722801925541 state 0x0 ref 1 00000100:00100000:2.0:1789760741.525619:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722802185957 time 722801925541 state 0x1 ref 1 00000100:00100000:2.0:1789760741.525644:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722802185957 time 722840948216 state 0x0 ref 1 00000100:00100000:2.0:1789760741.525658:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507242112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.525751:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19562496, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.525762:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.525781:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.525795:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.525803:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.525821:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.567217:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972096 is committed 00002000:00010000:0.0:1789760741.567329:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19562496 -> 19566592 00010000:00010000:0.0:1789760741.567349:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567365:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567374:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567383:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760741.567389:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567399:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567489:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.567674:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7ed3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.568409:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507242112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42746us (43432us total) trans 4294972096 rc 0/0 00000100:00100000:0.0:1789760741.568423:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4806 00000100:00100000:0.0:1789760741.568436:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722802185957 time 722840948216 state 0x2 ref 0 00000100:00100000:0.0:1789760741.573405:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.573470:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242240 00000100:00100000:2.0:1789760741.573483:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242240 00000100:00100000:2.0:1789760741.573563:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722802185957 time 722840948216 state 0x0 ref 1 00000100:00100000:2.0:1789760741.573580:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722841208632 time 722840948216 state 0x1 ref 1 00000100:00100000:2.0:1789760741.573614:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722841208632 time 722888916655 state 0x0 ref 1 00000100:00100000:2.0:1789760741.573627:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507242240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.573712:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19566592, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.573723:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.573743:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760741.573748:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.573756:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.573774:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.609640:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19566592 -> 19570688 00010000:00010000:2.0:1789760741.609666:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609683:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609739:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609747:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.609753:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609763:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609797:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.609805:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.609943:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507242240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36309us (36545us total) trans 4294972097 rc 0/0 00000100:00100000:2.0:1789760741.609954:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4807 00000100:00100000:2.0:1789760741.609966:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722841208632 time 722888916655 state 0x2 ref 0 00000001:00080000:1.0:1789760741.610121:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972097 is committed 00000100:00100000:2.0:1789760741.613341:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760741.613482:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242368 00000100:00100000:3.0:1789760741.613495:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242368 00000100:00100000:3.0:1789760741.613568:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722841208632 time 722888916655 state 0x0 ref 1 00000100:00100000:3.0:1789760741.613588:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722889177071 time 722888916655 state 0x1 ref 1 00000100:00100000:3.0:1789760741.613626:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722889177071 time 722928928829 state 0x0 ref 1 00000100:00100000:3.0:1789760741.613641:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760741.613739:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19570688, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:3.0:1789760741.613751:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.613770:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760741.613776:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.613784:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.613813:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.661638:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972098 is committed 00002000:00010000:2.0:1789760741.661837:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19570688 -> 19574784 00010000:00010000:2.0:1789760741.661856:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661879:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661889:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661896:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.661902:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661913:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661937:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.661944:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b7ee1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.662092:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48447us (48757us total) trans 4294972098 rc 0/0 00000100:00100000:2.0:1789760741.662109:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4808 00000100:00100000:2.0:1789760741.662122:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722889177071 time 722928928829 state 0x2 ref 0 00000100:00100000:3.0:1789760741.665455:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760741.665551:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242496 00000100:00100000:0.0:1789760741.665562:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242496 00000100:00100000:0.0:1789760741.665629:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722889177071 time 722928928829 state 0x0 ref 1 00000100:00100000:0.0:1789760741.665647:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722929189245 time 722928928829 state 0x1 ref 1 00000100:00100000:0.0:1789760741.665796:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722929189245 time 722981089725 state 0x0 ref 1 00000100:00100000:0.0:1789760741.665811:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507242496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760741.665881:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19574784, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:0.0:1789760741.665891:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.665910:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760741.665915:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.665922:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.665939:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.702611:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972099 is committed 00002000:00010000:2.0:1789760741.702965:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19574784 -> 19578880 00010000:00010000:2.0:1789760741.702986:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703002:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703013:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703020:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760741.703027:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703037:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703069:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.703078:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b7ee8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.703332:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507242496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37515us (37881us total) trans 4294972099 rc 0/0 00000100:00100000:2.0:1789760741.703346:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4809 00000100:00100000:2.0:1789760741.703359:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722929189245 time 722981089725 state 0x2 ref 0 00000100:00100000:1.0:1789760741.712415:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.713028:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242624 00000100:00100000:2.0:1789760741.713043:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242624 00000100:00100000:2.0:1789760741.713123:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722929189245 time 722981089725 state 0x0 ref 1 00000100:00100000:2.0:1789760741.713142:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722981350141 time 722981089725 state 0x1 ref 1 00000100:00100000:2.0:1789760741.713181:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722981350141 time 723028484027 state 0x0 ref 1 00000100:00100000:2.0:1789760741.713194:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.713263:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19578880, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.713274:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.713293:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760741.713298:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.713305:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.713322:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.757953:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972100 is committed 00002000:00010000:1.0:1789760741.758126:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19578880 -> 19582976 00010000:00010000:1.0:1789760741.758146:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758161:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758170:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758178:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.758184:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758193:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758217:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.758225:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b7eef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.758350:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45151us (45943us total) trans 4294972100 rc 0/0 00000100:00100000:1.0:1789760741.758362:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4810 00000100:00100000:1.0:1789760741.758374:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722981350141 time 723028484027 state 0x2 ref 0 00000100:00100000:0.0:1789760741.763126:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760741.763199:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242752 00000100:00100000:2.0:1789760741.763214:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242752 00000100:00100000:2.0:1789760741.763291:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 722981350141 time 723028484027 state 0x0 ref 1 00000100:00100000:2.0:1789760741.763321:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723028744443 time 723028484027 state 0x1 ref 1 00000100:00100000:2.0:1789760741.763357:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723028744443 time 723078659518 state 0x0 ref 1 00000100:00100000:2.0:1789760741.763373:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760741.763544:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19582976, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760741.763555:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.763575:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760741.763580:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.763587:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.763603:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.801189:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972101 is committed 00002000:00010000:1.0:1789760741.801326:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19582976 -> 19587072 00010000:00010000:1.0:1789760741.801347:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801363:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801372:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801380:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.801386:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801396:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801422:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.801430:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b7ef6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.801554:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507242752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38177us (38436us total) trans 4294972101 rc 0/0 00000100:00100000:1.0:1789760741.801565:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4811 00000100:00100000:1.0:1789760741.801577:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723028744443 time 723078659518 state 0x2 ref 0 00000100:00100000:2.0:1789760741.804944:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760741.805117:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507242880 00000100:00100000:3.0:1789760741.805132:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507242880 00000100:00100000:3.0:1789760741.805203:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723028744443 time 723078659518 state 0x0 ref 1 00000100:00100000:3.0:1789760741.805222:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723078919934 time 723078659518 state 0x1 ref 1 00000100:00100000:3.0:1789760741.805260:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723078919934 time 723120561999 state 0x0 ref 1 00000100:00100000:3.0:1789760741.805275:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507242880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760741.805343:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19587072, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:3.0:1789760741.805355:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.805377:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760741.805383:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.805391:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.805409:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.839574:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972102 is committed 00002000:00010000:1.0:1789760741.839678:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19587072 -> 19591168 00010000:00010000:1.0:1789760741.839772:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839794:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839804:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839812:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.839817:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839834:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839854:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.839862:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b7efd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.839956:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507242880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34678us (35020us total) trans 4294972102 rc 0/0 00000100:00100000:1.0:1789760741.839969:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4812 00000100:00100000:1.0:1789760741.839977:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723078919934 time 723120561999 state 0x2 ref 0 00000100:00100000:2.0:1789760741.843551:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760741.844327:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243008 00000100:00100000:1.0:1789760741.844338:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243008 00000100:00100000:1.0:1789760741.844394:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723078919934 time 723120561999 state 0x0 ref 1 00000100:00100000:1.0:1789760741.844409:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723120822415 time 723120561999 state 0x1 ref 1 00000100:00100000:1.0:1789760741.844444:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723120822415 time 723159747175 state 0x0 ref 1 00000100:00100000:1.0:1789760741.844466:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760741.844517:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19591168, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:1.0:1789760741.844526:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.844543:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760741.844547:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.844554:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.844570:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.878293:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972103 is committed 00002000:00010000:1.0:1789760741.878579:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19591168 -> 19595264 00010000:00010000:1.0:1789760741.878599:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878616:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878639:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878648:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.878654:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878664:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878689:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.878726:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.878857:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34385us (35308us total) trans 4294972103 rc 0/0 00000100:00100000:1.0:1789760741.878870:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4813 00000100:00100000:1.0:1789760741.878882:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723120822415 time 723159747175 state 0x2 ref 0 00000100:00100000:0.0:1789760741.882168:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760741.882323:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243136 00000100:00100000:1.0:1789760741.882338:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243136 00000100:00100000:1.0:1789760741.882413:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723120822415 time 723159747175 state 0x0 ref 1 00000100:00100000:1.0:1789760741.882433:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723160007591 time 723159747175 state 0x1 ref 1 00000100:00100000:1.0:1789760741.882507:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723160007591 time 723197809921 state 0x0 ref 1 00000100:00100000:1.0:1789760741.882523:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21face450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760741.882590:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19595264, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:1.0:1789760741.882603:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.882623:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760741.882628:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.882636:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.882654:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760741.917071:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19595264 -> 19599360 00000001:00080000:2.0:1789760741.917097:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972104 is committed 00010000:00010000:1.0:1789760741.917102:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917117:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917134:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917142:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760741.917148:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917158:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917182:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.917190:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b7f0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.917386:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21face450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34858us (35225us total) trans 4294972104 rc 0/0 00000100:00100000:1.0:1789760741.917399:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4814 00000100:00100000:1.0:1789760741.917413:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723160007591 time 723197809921 state 0x2 ref 0 00000100:00100000:2.0:1789760741.921450:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760741.921819:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243264 00000100:00100000:1.0:1789760741.921901:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243264 00000100:00100000:0.0:1789760741.922051:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723160007591 time 723197809921 state 0x0 ref 1 00000100:00100000:0.0:1789760741.922072:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723198070337 time 723197809921 state 0x1 ref 1 00000100:00100000:0.0:1789760741.922159:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723198070337 time 723237460809 state 0x0 ref 1 00000100:00100000:0.0:1789760741.922179:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760741.922258:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19599360, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:0.0:1789760741.922270:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.922291:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760741.922297:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.922305:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.922324:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.955856:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19599360 -> 19603456 00010000:00010000:3.0:1789760741.955882:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955898:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955909:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955916:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760741.955922:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955933:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955957:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.955966:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b7f12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.956086:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507243264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33903us (34646us total) trans 4294972105 rc 0/0 00000100:00100000:3.0:1789760741.956102:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4815 00000100:00100000:3.0:1789760741.956114:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723198070337 time 723237460809 state 0x2 ref 0 00000001:00080000:2.0:1789760741.956391:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972105 is committed 00000100:00100000:2.0:1789760741.960372:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760741.961012:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243392 00000100:00100000:1.0:1789760741.961027:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243392 00000100:00100000:1.0:1789760741.961103:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723198070337 time 723237460809 state 0x0 ref 1 00000100:00100000:1.0:1789760741.961124:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723237721225 time 723237460809 state 0x1 ref 1 00000100:00100000:1.0:1789760741.961148:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723237721225 time 723276452079 state 0x0 ref 1 00000100:00100000:1.0:1789760741.961162:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760741.961225:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19603456, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:1.0:1789760741.961236:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.961255:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760741.961260:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.961267:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.961284:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.005064:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972106 is committed 00002000:00010000:2.0:1789760742.005151:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19603456 -> 19607552 00010000:00010000:2.0:1789760742.005171:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005193:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005203:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005210:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760742.005216:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005227:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005253:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.005262:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b7f19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.005451:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44283us (45087us total) trans 4294972106 rc 0/0 00000100:00100000:2.0:1789760742.005464:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4816 00000100:00100000:2.0:1789760742.005477:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723237721225 time 723276452079 state 0x2 ref 0 00000100:00100000:1.0:1789760742.009654:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760742.010067:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243520 00000100:00100000:2.0:1789760742.010082:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243520 00000100:00100000:2.0:1789760742.010159:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723237721225 time 723276452079 state 0x0 ref 1 00000100:00100000:2.0:1789760742.010179:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723276712495 time 723276452079 state 0x1 ref 1 00000100:00100000:2.0:1789760742.010218:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723276712495 time 723325521428 state 0x0 ref 1 00000100:00100000:2.0:1789760742.010234:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760742.010300:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19607552, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0x9807 00010000:00010000:2.0:1789760742.010311:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.010331:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760742.010337:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.010345:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.010360:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.054609:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972107 is committed 00002000:00010000:2.0:1789760742.054740:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19607552 -> 19611648 00002000:00010000:2.0:1789760742.054746:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:2.0:1789760742.054749:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:2.0:1789760742.054765:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054780:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054802:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054810:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760742.054816:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054833:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054857:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.054866:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b7f20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.054980:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44742us (45337us total) trans 4294972107 rc 0/0 00000100:00100000:2.0:1789760742.054991:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4817 00000100:00100000:2.0:1789760742.055004:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723276712495 time 723325521428 state 0x2 ref 0 00000100:00100000:3.0:1789760742.058471:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760742.058807:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243648 00000100:00100000:0.0:1789760742.058821:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243648 00000100:00100000:0.0:1789760742.058897:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723276712495 time 723325521428 state 0x0 ref 1 00000100:00100000:0.0:1789760742.058917:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723325781844 time 723325521428 state 0x1 ref 1 00000100:00100000:0.0:1789760742.059007:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723325781844 time 723374309051 state 0x0 ref 1 00000100:00100000:0.0:1789760742.059021:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760742.059092:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19611648, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:0.0:1789760742.059105:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059144:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760742.059150:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.059159:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059178:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.120325:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972108 is committed 00002000:00010000:1.0:1789760742.120436:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19611648 -> 19615744 00010000:00010000:1.0:1789760742.120458:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120474:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120491:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120499:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.120504:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120514:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120538:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.120546:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b7f27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.120743:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507243648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61671us (62234us total) trans 4294972108 rc 0/0 00000100:00100000:1.0:1789760742.120755:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4818 00000100:00100000:1.0:1789760742.120768:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723325781844 time 723374309051 state 0x2 ref 0 00000100:00100000:1.0:1789760742.125035:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760742.125103:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243776 00000100:00100000:2.0:1789760742.125115:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243776 00000100:00100000:2.0:1789760742.125187:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723325781844 time 723374309051 state 0x0 ref 1 00000100:00100000:2.0:1789760742.125208:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723374569467 time 723374309051 state 0x1 ref 1 00000100:00100000:2.0:1789760742.125322:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723374569467 time 723440624156 state 0x0 ref 1 00000100:00100000:2.0:1789760742.125338:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760742.125404:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19615744, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:2.0:1789760742.125415:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.125437:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760742.125442:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.125450:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.125466:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.160554:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19615744 -> 19619840 00010000:00010000:1.0:1789760742.160581:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160603:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160614:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160623:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.160629:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160640:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160664:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.160672:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b7f2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.161011:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972109 is committed 00000100:00100000:1.0:1789760742.161074:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35730us (36045us total) trans 4294972109 rc 0/0 00000100:00100000:1.0:1789760742.161088:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4819 00000100:00100000:1.0:1789760742.161101:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723374569467 time 723440624156 state 0x2 ref 0 00000100:00100000:2.0:1789760742.165445:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760742.165547:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507243904 00000100:00100000:1.0:1789760742.165560:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507243904 00000100:00100000:1.0:1789760742.165630:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723374569467 time 723440624156 state 0x0 ref 1 00000100:00100000:1.0:1789760742.165646:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723440884572 time 723440624156 state 0x1 ref 1 00000100:00100000:1.0:1789760742.165673:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723440884572 time 723480976259 state 0x0 ref 1 00000100:00100000:1.0:1789760742.165687:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760742.165777:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19619840, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:1.0:1789760742.165800:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.165921:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760742.165928:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.165936:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.165951:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.211221:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972110 is committed 00002000:00010000:3.0:1789760742.211660:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19619840 -> 19623936 00010000:00010000:3.0:1789760742.211679:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211719:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211728:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211736:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.211742:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211751:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211775:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.211793:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b7f35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.211921:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507243904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46228us (46482us total) trans 4294972110 rc 0/0 00000100:00100000:3.0:1789760742.211933:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4820 00000100:00100000:3.0:1789760742.211946:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723440884572 time 723480976259 state 0x2 ref 0 00000100:00100000:1.0:1789760742.219083:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760742.219205:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244032 00000100:00100000:0.0:1789760742.219220:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244032 00000100:00100000:0.0:1789760742.219299:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723440884572 time 723480976259 state 0x0 ref 1 00000100:00100000:0.0:1789760742.219319:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723481236675 time 723480976259 state 0x1 ref 1 00000100:00100000:0.0:1789760742.219355:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723481236675 time 723534658148 state 0x0 ref 1 00000100:00100000:0.0:1789760742.219370:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507244032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760742.219435:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19623936, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:0.0:1789760742.219446:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.219466:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760742.219472:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.219479:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.219498:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.267939:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972111 is committed 00002000:00010000:1.0:1789760742.268093:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19623936 -> 19628032 00010000:00010000:1.0:1789760742.268112:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268128:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268144:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268153:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.268158:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268168:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268388:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.268399:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b7f3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.268534:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507244032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49160us (49458us total) trans 4294972111 rc 0/0 00000100:00100000:1.0:1789760742.268652:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4821 00000100:00100000:1.0:1789760742.268665:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723481236675 time 723534658148 state 0x2 ref 0 00000100:00100000:0.0:1789760742.278249:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760742.281350:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244160 00000100:00100000:1.0:1789760742.281375:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244160 00000100:00100000:1.0:1789760742.281468:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723481236675 time 723534658148 state 0x0 ref 1 00000100:00100000:1.0:1789760742.281489:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723534918564 time 723534658148 state 0x1 ref 1 00000100:00100000:1.0:1789760742.281674:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723534918564 time 723596975410 state 0x0 ref 1 00000100:00100000:1.0:1789760742.281730:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760742.281809:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19628032, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:1.0:1789760742.281821:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.281839:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760742.281845:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.281853:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.281871:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.300086:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760742.336665:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19628032 -> 19632128 00010000:00010000:1.0:1789760742.336723:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336740:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336751:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336760:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.336766:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336777:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336815:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.336891:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.337016:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507244160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55320us (58844us total) trans 4294972112 rc 0/0 00000100:00100000:1.0:1789760742.337029:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4822 00000100:00100000:1.0:1789760742.337042:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723534918564 time 723596975410 state 0x2 ref 0 00000001:00080000:3.0:1789760742.337143:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972112 is committed 00000100:00100000:0.0:1789760742.342876:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.342947:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244288 00000100:00100000:3.0:1789760742.342961:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244288 00000100:00100000:3.0:1789760742.343034:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723534918564 time 723596975410 state 0x0 ref 1 00000100:00100000:3.0:1789760742.343054:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723597235826 time 723596975410 state 0x1 ref 1 00000100:00100000:3.0:1789760742.343090:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723597235826 time 723658392995 state 0x0 ref 1 00000100:00100000:3.0:1789760742.343105:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507244288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.343174:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19632128, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.343185:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.343205:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760742.343210:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.343219:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.343255:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.381728:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19632128 -> 19636224 00010000:00010000:3.0:1789760742.381756:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381772:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381794:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381803:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.381809:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381825:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381850:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.381858:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.381976:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507244288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38867us (39107us total) trans 4294972113 rc 0/0 00000100:00100000:3.0:1789760742.381988:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4823 00000100:00100000:3.0:1789760742.382000:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723597235826 time 723658392995 state 0x2 ref 0 00000001:00080000:2.0:1789760742.382598:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972113 is committed 00000100:00100000:2.0:1789760742.390820:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760742.390976:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244416 00000100:00100000:1.0:1789760742.390989:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244416 00000100:00100000:1.0:1789760742.391065:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723597235826 time 723658392995 state 0x0 ref 1 00000100:00100000:1.0:1789760742.391084:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723658653411 time 723658392995 state 0x1 ref 1 00000100:00100000:1.0:1789760742.391121:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723658653411 time 723706423329 state 0x0 ref 1 00000100:00100000:1.0:1789760742.391136:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760742.391204:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19636224, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:1.0:1789760742.391216:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.391237:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760742.391244:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.391251:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.391268:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.435225:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972114 is committed 00002000:00010000:3.0:1789760742.435294:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19636224 -> 19640320 00010000:00010000:3.0:1789760742.435317:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435333:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435343:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435350:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.435355:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435365:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435476:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.435491:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b7f51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.435650:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44510us (44839us total) trans 4294972114 rc 0/0 00000100:00100000:3.0:1789760742.435674:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4824 00000100:00100000:3.0:1789760742.435688:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723658653411 time 723706423329 state 0x2 ref 0 00000100:00100000:0.0:1789760742.439618:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760742.439762:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244544 00000100:00100000:1.0:1789760742.439777:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244544 00000100:00100000:1.0:1789760742.440033:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723658653411 time 723706423329 state 0x0 ref 1 00000100:00100000:1.0:1789760742.440052:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723706683745 time 723706423329 state 0x1 ref 1 00000100:00100000:1.0:1789760742.440089:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723706683745 time 723755390876 state 0x0 ref 1 00000100:00100000:1.0:1789760742.440106:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760742.440230:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19640320, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:1.0:1789760742.440241:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.440261:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760742.440267:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.440275:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.440293:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.495175:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972115 is committed 00002000:00010000:1.0:1789760742.495335:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19640320 -> 19644416 00010000:00010000:1.0:1789760742.495357:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495373:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495384:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495392:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.495398:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495409:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495434:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.495443:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.495584:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55475us (55973us total) trans 4294972115 rc 0/0 00000100:00100000:1.0:1789760742.495596:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4825 00000100:00100000:1.0:1789760742.495616:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723706683745 time 723755390876 state 0x2 ref 0 00000100:00100000:0.0:1789760742.503356:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.503461:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244672 00000100:00100000:3.0:1789760742.503474:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244672 00000100:00100000:3.0:1789760742.503617:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723706683745 time 723755390876 state 0x0 ref 1 00000100:00100000:3.0:1789760742.503638:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723755651292 time 723755390876 state 0x1 ref 1 00000100:00100000:3.0:1789760742.504089:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723755651292 time 723819388802 state 0x0 ref 1 00000100:00100000:3.0:1789760742.504108:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507244672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.504172:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19644416, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.504183:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.504201:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760742.504206:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.504213:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.504233:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.550270:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972116 is committed 00002000:00010000:3.0:1789760742.551222:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19644416 -> 19648512 00010000:00010000:3.0:1789760742.551462:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551480:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551492:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551501:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.551508:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551518:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551542:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.551550:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.551732:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507244672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47581us (48343us total) trans 4294972116 rc 0/0 00000100:00100000:3.0:1789760742.551746:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4826 00000100:00100000:3.0:1789760742.551758:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723755651292 time 723819388802 state 0x2 ref 0 00000100:00100000:2.0:1789760742.555807:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.556138:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244800 00000100:00100000:3.0:1789760742.556151:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244800 00000100:00100000:3.0:1789760742.556229:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723755651292 time 723819388802 state 0x0 ref 1 00000100:00100000:3.0:1789760742.556248:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723819649218 time 723819388802 state 0x1 ref 1 00000100:00100000:3.0:1789760742.556532:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723819649218 time 723871832168 state 0x0 ref 1 00000100:00100000:3.0:1789760742.556547:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507244800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.556613:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19648512, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.556625:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.556644:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760742.556649:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.556657:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.556674:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.588146:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972117 is committed 00002000:00010000:2.0:1789760742.588370:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19648512 -> 19652608 00010000:00010000:2.0:1789760742.588390:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588412:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588423:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588431:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760742.588437:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588448:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588473:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588482:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b7f66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.588601:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507244800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32050us (32802us total) trans 4294972117 rc 0/0 00000100:00100000:2.0:1789760742.588614:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4827 00000100:00100000:2.0:1789760742.588627:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723819649218 time 723871832168 state 0x2 ref 0 00000100:00100000:0.0:1789760742.595311:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760742.595523:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507244928 00000100:00100000:1.0:1789760742.595535:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507244928 00000100:00100000:1.0:1789760742.595603:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723819649218 time 723871832168 state 0x0 ref 1 00000100:00100000:1.0:1789760742.595622:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723872092584 time 723871832168 state 0x1 ref 1 00000100:00100000:1.0:1789760742.595663:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723872092584 time 723910965687 state 0x0 ref 1 00000100:00100000:1.0:1789760742.595677:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760742.595792:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19652608, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:1.0:1789760742.595804:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.596280:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760742.596288:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.596296:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.596314:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.650771:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972118 is committed 00002000:00010000:3.0:1789760742.651130:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19652608 -> 19656704 00010000:00010000:3.0:1789760742.651149:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651165:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651177:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651185:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.651191:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651202:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651227:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.651234:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b7f6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.651385:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507244928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55702us (56080us total) trans 4294972118 rc 0/0 00000100:00100000:3.0:1789760742.651402:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4828 00000100:00100000:3.0:1789760742.651415:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723872092584 time 723910965687 state 0x2 ref 0 00000100:00100000:0.0:1789760742.659291:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.659412:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245056 00000100:00100000:3.0:1789760742.659425:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245056 00000100:00100000:3.0:1789760742.659500:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723872092584 time 723910965687 state 0x0 ref 1 00000100:00100000:3.0:1789760742.659518:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723911226103 time 723910965687 state 0x1 ref 1 00000100:00100000:3.0:1789760742.659612:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723911226103 time 723974913821 state 0x0 ref 1 00000100:00100000:3.0:1789760742.659627:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.659689:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19656704, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.659738:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.659760:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760742.659766:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.659790:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.659809:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.695283:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972119 is committed 00002000:00010000:2.0:1789760742.695441:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19656704 -> 19660800 00010000:00010000:2.0:1789760742.695461:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695474:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695484:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695492:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760742.695496:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695506:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695528:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.695537:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b7f74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.695632:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36001us (36405us total) trans 4294972119 rc 0/0 00000100:00100000:2.0:1789760742.695642:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4829 00000100:00100000:2.0:1789760742.695652:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723911226103 time 723974913821 state 0x2 ref 0 00000100:00100000:2.0:1789760742.702437:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760742.702520:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245184 00000100:00100000:0.0:1789760742.702539:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245184 00000100:00100000:0.0:1789760742.702631:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723911226103 time 723974913821 state 0x0 ref 1 00000100:00100000:0.0:1789760742.702651:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723975174237 time 723974913821 state 0x1 ref 1 00000100:00100000:0.0:1789760742.702771:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723975174237 time 724018072337 state 0x0 ref 1 00000100:00100000:0.0:1789760742.702799:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760742.702867:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19660800, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:0.0:1789760742.702878:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.702898:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760742.702904:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.702911:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.702928:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.740411:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972120 is committed 00002000:00010000:3.0:1789760742.741252:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19660800 -> 19664896 00010000:00010000:3.0:1789760742.741274:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741293:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741304:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741312:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.741317:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741378:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741406:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.741416:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b7f7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.741651:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38849us (39216us total) trans 4294972120 rc 0/0 00000100:00100000:3.0:1789760742.741665:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4830 00000100:00100000:3.0:1789760742.741676:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723975174237 time 724018072337 state 0x2 ref 0 00000100:00100000:0.0:1789760742.746968:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.747133:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245312 00000100:00100000:3.0:1789760742.747147:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245312 00000100:00100000:2.0:1789760742.747327:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 723975174237 time 724018072337 state 0x0 ref 1 00000100:00100000:2.0:1789760742.747349:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724018332753 time 724018072337 state 0x1 ref 1 00000100:00100000:2.0:1789760742.747429:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724018332753 time 724062731004 state 0x0 ref 1 00000100:00100000:2.0:1789760742.747445:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507245312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760742.747510:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19664896, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:2.0:1789760742.747522:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.747542:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760742.747549:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.747556:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.747574:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.781194:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972121 is committed 00002000:00010000:3.0:1789760742.781319:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19664896 -> 19668992 00010000:00010000:3.0:1789760742.781340:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781356:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781367:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781374:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.781380:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781390:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781414:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.781421:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b7f82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.781540:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507245312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34092us (34580us total) trans 4294972121 rc 0/0 00000100:00100000:3.0:1789760742.781553:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4831 00000100:00100000:3.0:1789760742.781565:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724018332753 time 724062731004 state 0x2 ref 0 00000100:00100000:0.0:1789760742.785323:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760742.785441:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245440 00000100:00100000:2.0:1789760742.785455:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245440 00000100:00100000:2.0:1789760742.785533:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724018332753 time 724062731004 state 0x0 ref 1 00000100:00100000:2.0:1789760742.785553:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724062991420 time 724062731004 state 0x1 ref 1 00000100:00100000:2.0:1789760742.785648:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724062991420 time 724100950652 state 0x0 ref 1 00000100:00100000:2.0:1789760742.785663:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507245440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760742.785789:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19668992, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:2.0:1789760742.785801:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.785830:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760742.785837:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.785845:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.785863:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.850305:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19668992 -> 19673088 00010000:00010000:3.0:1789760742.850334:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850349:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850358:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850366:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.850371:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850380:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850404:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.850411:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b7f89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.850540:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507245440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64872us (65224us total) trans 4294972122 rc 0/0 00000100:00100000:3.0:1789760742.850555:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4832 00000100:00100000:3.0:1789760742.850567:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724062991420 time 724100950652 state 0x2 ref 0 00000001:00080000:1.0:1789760742.850910:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972122 is committed 00000100:00100000:2.0:1789760742.854339:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.855442:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245568 00000100:00100000:3.0:1789760742.855459:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245568 00000100:00100000:3.0:1789760742.855537:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724062991420 time 724100950652 state 0x0 ref 1 00000100:00100000:3.0:1789760742.855557:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724101211068 time 724100950652 state 0x1 ref 1 00000100:00100000:3.0:1789760742.855590:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724101211068 time 724170892808 state 0x0 ref 1 00000100:00100000:3.0:1789760742.855604:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507245568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.855837:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19673088, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.855849:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.855871:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760742.855876:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.855884:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.855901:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.899653:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972123 is committed 00002000:00010000:1.0:1789760742.899798:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19673088 -> 19677184 00010000:00010000:1.0:1789760742.899825:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899842:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899852:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899860:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760742.899866:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899876:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899900:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.899908:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7f90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.900073:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507245568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44464us (45742us total) trans 4294972123 rc 0/0 00000100:00100000:1.0:1789760742.900086:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4833 00000100:00100000:1.0:1789760742.900098:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724101211068 time 724170892808 state 0x2 ref 0 00000100:00100000:2.0:1789760742.903048:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.903146:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245696 00000100:00100000:3.0:1789760742.903159:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245696 00000100:00100000:3.0:1789760742.903232:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724101211068 time 724170892808 state 0x0 ref 1 00000100:00100000:3.0:1789760742.903251:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724171153224 time 724170892808 state 0x1 ref 1 00000100:00100000:3.0:1789760742.903284:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724171153224 time 724218587773 state 0x0 ref 1 00000100:00100000:3.0:1789760742.903298:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507245696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.903363:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19677184, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.903373:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.903392:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760742.903397:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.903404:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.903420:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.935288:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19677184 -> 19681280 00010000:00010000:3.0:1789760742.935318:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935337:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935348:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935358:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.935364:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935375:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935401:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.935410:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b7f97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.935539:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507245696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32236us (32500us total) trans 4294972124 rc 0/0 00000100:00100000:3.0:1789760742.935552:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4834 00000100:00100000:3.0:1789760742.935564:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724171153224 time 724218587773 state 0x2 ref 0 00000001:00080000:2.0:1789760742.935647:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972124 is committed 00000100:00100000:2.0:1789760742.941514:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.941610:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245824 00000100:00100000:3.0:1789760742.941625:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245824 00000100:00100000:3.0:1789760742.941735:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724171153224 time 724218587773 state 0x0 ref 1 00000100:00100000:3.0:1789760742.941756:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724218848189 time 724218587773 state 0x1 ref 1 00000100:00100000:3.0:1789760742.941848:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724218848189 time 724257149366 state 0x0 ref 1 00000100:00100000:3.0:1789760742.941869:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.941934:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19681280, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.941945:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.941965:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760742.941970:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.941978:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.941995:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.981653:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972125 is committed 00002000:00010000:3.0:1789760742.981809:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19681280 -> 19685376 00010000:00010000:3.0:1789760742.981886:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981902:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981912:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981920:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760742.981926:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981937:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981963:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.981972:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7f9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.982096:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40224us (40595us total) trans 4294972125 rc 0/0 00000100:00100000:3.0:1789760742.982109:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4835 00000100:00100000:3.0:1789760742.982121:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724218848189 time 724257149366 state 0x2 ref 0 00000100:00100000:1.0:1789760742.989064:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760742.989547:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507245952 00000100:00100000:3.0:1789760742.989563:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507245952 00000100:00100000:3.0:1789760742.989641:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724218848189 time 724257149366 state 0x0 ref 1 00000100:00100000:3.0:1789760742.989663:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724257409782 time 724257149366 state 0x1 ref 1 00000100:00100000:3.0:1789760742.989769:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724257409782 time 724305071185 state 0x0 ref 1 00000100:00100000:3.0:1789760742.989793:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760742.989859:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19685376, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:3.0:1789760742.989871:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.989892:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760742.989898:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.989905:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.989922:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.027661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972126 is committed 00002000:00010000:0.0:1789760743.027829:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19685376 -> 19689472 00010000:00010000:0.0:1789760743.027852:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027867:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027879:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027887:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.027893:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027903:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027931:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.027940:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.028074:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507245952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38277us (39028us total) trans 4294972126 rc 0/0 00000100:00100000:0.0:1789760743.028086:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4836 00000100:00100000:0.0:1789760743.028098:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724257409782 time 724305071185 state 0x2 ref 0 00000100:00100000:3.0:1789760743.036044:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.036404:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246080 00000100:00100000:2.0:1789760743.036417:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246080 00000100:00100000:2.0:1789760743.036514:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724257409782 time 724305071185 state 0x0 ref 1 00000100:00080000:3.0:1789760743.036532:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760743.036534:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724305331601 time 724305071185 state 0x1 ref 1 00000100:00100000:2.0:1789760743.036622:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724305331601 time 724351923938 state 0x0 ref 1 00000100:00100000:2.0:1789760743.036639:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507246080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.036744:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19689472, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0x9807 00010000:00010000:2.0:1789760743.036755:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.036784:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.036790:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.036799:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.036817:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.075036:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972127 is committed 00002000:00010000:2.0:1789760743.075133:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19689472 -> 19693568 00002000:00010000:2.0:1789760743.075138:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:2.0:1789760743.075141:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:2.0:1789760743.075158:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075173:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075183:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075196:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.075202:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075211:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075235:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.075245:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.075487:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507246080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38845us (39454us total) trans 4294972127 rc 0/0 00000100:00100000:2.0:1789760743.075500:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4837 00000100:00100000:2.0:1789760743.075511:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724305331601 time 724351923938 state 0x2 ref 0 00000100:00100000:3.0:1789760743.080643:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.081189:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246208 00000100:00100000:2.0:1789760743.081203:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246208 00000100:00100000:2.0:1789760743.081315:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724305331601 time 724351923938 state 0x0 ref 1 00000100:00100000:2.0:1789760743.081335:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724352184354 time 724351923938 state 0x1 ref 1 00000100:00100000:2.0:1789760743.081533:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724352184354 time 724396834873 state 0x0 ref 1 00000100:00100000:2.0:1789760743.081553:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507246208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.081617:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19693568, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.081628:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.081649:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.081655:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.081662:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.081680:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.128669:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.128753:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.128755:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246464 00000100:00100000:0.0:1789760743.128788:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246464 00000100:00100000:3.0:1789760743.128795:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.128827:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.128977:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.129245:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246336 00000100:00100000:2.0:1789760743.129251:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246336 00000100:00100000:2.0:1789760743.129302:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 468 00000100:00100000:2.0:1789760743.129329:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223495450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507246336:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760743.129426:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223495450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507246336:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 113us (672us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760743.129435:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 468 00000100:00100000:0.0:1789760743.131225:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246592 00000100:00100000:0.0:1789760743.131243:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246592 00000100:00100000:1.0:1789760743.131282:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246848 00000100:00100000:1.0:1789760743.131288:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246848 00000100:00100000:0.0:1789760743.131307:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246720 00000100:00100000:0.0:1789760743.131310:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246720 00000100:00100000:1.0:1789760743.131320:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 888 00000100:00100000:0.0:1789760743.131329:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 889 00000100:00100000:1.0:1789760743.131330:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507246464:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760743.131335:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507246592:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760743.131414:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 890 00000100:00100000:3.0:1789760743.131425:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507246848:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760743.131464:0:7721:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 5 min 3 max 116 00000100:00100000:2.0:1789760743.131475:0:7721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt00_005' 00000100:00080000:2.0:1789760743.131501:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760743.131506:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760743.131639:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 891 00000100:00100000:2.0:1789760743.131649:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507246720:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760743.131732:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507246592:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 395us (2934us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760743.131741:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 889 00000100:00100000:1.0:1789760743.131791:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507246464:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 462us (3136us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760743.131798:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 888 00000100:00100000:2.0:1789760743.131966:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507246720:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 318us (2988us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760743.131974:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 891 00000100:00100000:3.0:1789760743.132408:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507246848:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 982us (3576us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.132417:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 890 00000001:00080000:0.0:1789760743.137644:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972128 is committed 00002000:00010000:2.0:1789760743.137793:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19693568 -> 19697664 00010000:00010000:2.0:1789760743.138099:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138116:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138125:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138132:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.138137:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138145:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138167:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.138173:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.138416:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507246208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56859us (57784us total) trans 4294972128 rc 0/0 00000100:00100000:2.0:1789760743.138428:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4838 00000100:00100000:2.0:1789760743.138439:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724352184354 time 724396834873 state 0x2 ref 0 00000100:00100000:3.0:1789760743.144410:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.144529:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507246976 00000100:00100000:0.0:1789760743.144542:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507246976 00000100:00100000:0.0:1789760743.144616:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724352184354 time 724396834873 state 0x0 ref 1 00000100:00100000:0.0:1789760743.144634:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724397095289 time 724396834873 state 0x1 ref 1 00000100:00100000:0.0:1789760743.144675:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724397095289 time 724459977986 state 0x0 ref 1 00000100:00100000:0.0:1789760743.144689:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507246976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760743.144798:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19697664, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:0.0:1789760743.144810:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.145011:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760743.145018:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.145026:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.145046:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.193931:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972129 is committed 00002000:00010000:0.0:1789760743.194029:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19697664 -> 19701760 00010000:00010000:0.0:1789760743.194050:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194067:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194077:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194090:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.194096:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194106:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194140:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.194149:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b7fba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.194294:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507246976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49600us (49893us total) trans 4294972129 rc 0/0 00000100:00100000:0.0:1789760743.194307:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4839 00000100:00100000:0.0:1789760743.194320:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724397095289 time 724459977986 state 0x2 ref 0 00000100:00100000:2.0:1789760743.200106:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.200339:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247104 00000100:00100000:3.0:1789760743.200355:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247104 00000100:00100000:3.0:1789760743.200423:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724397095289 time 724459977986 state 0x0 ref 1 00000100:00100000:3.0:1789760743.200442:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724460238402 time 724459977986 state 0x1 ref 1 00000100:00100000:3.0:1789760743.200477:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724460238402 time 724515780065 state 0x0 ref 1 00000100:00100000:3.0:1789760743.200490:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507247104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760743.200558:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19701760, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760743.200569:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.200590:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760743.200595:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.200604:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.200623:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.246032:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972130 is committed 00002000:00010000:2.0:1789760743.246169:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19701760 -> 19705856 00010000:00010000:2.0:1789760743.246190:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246205:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246220:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246229:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.246234:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246244:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246268:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.246283:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b7fc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.246408:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507247104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45911us (46309us total) trans 4294972130 rc 0/0 00000100:00100000:2.0:1789760743.246420:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4840 00000100:00100000:2.0:1789760743.246432:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724460238402 time 724515780065 state 0x2 ref 0 00000100:00100000:3.0:1789760743.251262:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.255147:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247232 00000100:00100000:2.0:1789760743.255169:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247232 00000100:00100000:2.0:1789760743.255265:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724460238402 time 724515780065 state 0x0 ref 1 00000100:00100000:2.0:1789760743.255287:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724516040481 time 724515780065 state 0x1 ref 1 00000100:00100000:2.0:1789760743.255328:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724516040481 time 724570631024 state 0x0 ref 1 00000100:00100000:2.0:1789760743.255344:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.255415:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19705856, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.255428:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.255450:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.255456:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.255464:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.255489:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.285880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972131 is committed 00002000:00010000:2.0:1789760743.285959:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19705856 -> 19709952 00010000:00010000:2.0:1789760743.285978:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.285990:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.285997:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.286002:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.286007:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.286019:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.286049:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.286058:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.286245:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30894us (34989us total) trans 4294972131 rc 0/0 00000100:00100000:2.0:1789760743.286258:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4841 00000100:00100000:2.0:1789760743.286271:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724516040481 time 724570631024 state 0x2 ref 0 00000100:00100000:3.0:1789760743.289118:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.290665:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247360 00000100:00100000:0.0:1789760743.290685:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247360 00000100:00100000:0.0:1789760743.290804:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724516040481 time 724570631024 state 0x0 ref 1 00000100:00100000:0.0:1789760743.290834:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724570891440 time 724570631024 state 0x1 ref 1 00000100:00100000:0.0:1789760743.291023:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724570891440 time 724606324677 state 0x0 ref 1 00000100:00100000:0.0:1789760743.291041:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507247360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760743.291109:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19709952, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:0.0:1789760743.291120:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.291145:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760743.291151:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.291160:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.291182:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.322066:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972132 is committed 00002000:00010000:0.0:1789760743.322250:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19709952 -> 19714048 00010000:00010000:0.0:1789760743.322269:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322284:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322295:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322302:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.322308:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322320:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322345:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.322352:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b7fcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.322589:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507247360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31543us (33476us total) trans 4294972132 rc 0/0 00000100:00100000:0.0:1789760743.322609:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4842 00000100:00100000:0.0:1789760743.322622:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724570891440 time 724606324677 state 0x2 ref 0 00010000:00010000:2.0:1789760743.323964:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760743.330167:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.330288:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247488 00000100:00100000:2.0:1789760743.330302:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247488 00000100:00100000:2.0:1789760743.330379:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724570891440 time 724606324677 state 0x0 ref 1 00000100:00100000:2.0:1789760743.330399:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724606585093 time 724606324677 state 0x1 ref 1 00000100:00100000:2.0:1789760743.330557:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724606585093 time 724645858529 state 0x0 ref 1 00000100:00100000:2.0:1789760743.330574:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.330651:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19714048, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.330662:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.330683:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.330689:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.330744:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.330763:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.381213:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19714048 -> 19718144 00010000:00010000:2.0:1789760743.381250:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.381250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972133 is committed 00010000:00010000:2.0:1789760743.381268:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.381278:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.381286:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.381292:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.381302:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.381340:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.381349:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b7fd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.381472:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50894us (51312us total) trans 4294972133 rc 0/0 00000100:00100000:2.0:1789760743.381487:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4843 00000100:00100000:2.0:1789760743.381510:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724606585093 time 724645858529 state 0x2 ref 0 00000100:00100000:0.0:1789760743.386919:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.387474:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247616 00000100:00100000:2.0:1789760743.387489:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247616 00000100:00100000:1.0:1789760743.387661:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724606585093 time 724645858529 state 0x0 ref 1 00000100:00100000:1.0:1789760743.387682:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724646118945 time 724645858529 state 0x1 ref 1 00000100:00100000:1.0:1789760743.387851:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724646118945 time 724703152898 state 0x0 ref 1 00000100:00100000:1.0:1789760743.387867:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760743.388115:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19718144, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:1.0:1789760743.388128:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.388154:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760743.388160:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.388170:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.388187:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.449619:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19718144 -> 19722240 00010000:00010000:3.0:1789760743.449644:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449656:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449664:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449669:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760743.449674:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449680:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449722:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.449731:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b7fdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.449782:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972134 is committed 00000100:00100000:3.0:1789760743.449874:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62003us (62963us total) trans 4294972134 rc 0/0 00000100:00100000:3.0:1789760743.449887:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4844 00000100:00100000:3.0:1789760743.449899:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724646118945 time 724703152898 state 0x2 ref 0 00000100:00100000:3.0:1789760743.455075:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.455673:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247744 00000100:00100000:0.0:1789760743.455689:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247744 00000100:00100000:0.0:1789760743.456091:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724646118945 time 724703152898 state 0x0 ref 1 00000100:00100000:0.0:1789760743.456112:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724703413314 time 724703152898 state 0x1 ref 1 00000100:00100000:0.0:1789760743.456605:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724703413314 time 724771904829 state 0x0 ref 1 00000100:00100000:0.0:1789760743.456625:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507247744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760743.457090:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19722240, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:0.0:1789760743.457105:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.457127:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760743.457134:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.457141:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.457160:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.499584:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19722240 -> 19726336 00010000:00010000:0.0:1789760743.499613:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499631:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499642:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499651:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.499656:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499667:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499717:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499728:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b7fe4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.499877:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507247744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43249us (44812us total) trans 4294972135 rc 0/0 00000100:00100000:0.0:1789760743.499890:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4845 00000100:00100000:0.0:1789760743.499903:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724703413314 time 724771904829 state 0x2 ref 0 00000001:00080000:3.0:1789760743.500492:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972135 is committed 00000100:00100000:0.0:1789760743.506461:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.506560:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507247872 00000100:00100000:2.0:1789760743.506573:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507247872 00000100:00100000:2.0:1789760743.506644:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724703413314 time 724771904829 state 0x0 ref 1 00000100:00100000:2.0:1789760743.506663:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724772165245 time 724771904829 state 0x1 ref 1 00000100:00100000:2.0:1789760743.506735:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724772165245 time 724822037679 state 0x0 ref 1 00000100:00100000:2.0:1789760743.506749:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507247872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.506827:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19726336, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.506838:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.506858:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.506864:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.506871:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.506887:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.570644:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972136 is committed 00002000:00010000:0.0:1789760743.570944:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19726336 -> 19730432 00010000:00010000:0.0:1789760743.570965:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.570981:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.570991:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.571000:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.571007:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.571023:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.571051:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.571059:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b7feb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.571264:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507247872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64511us (64811us total) trans 4294972136 rc 0/0 00000100:00100000:0.0:1789760743.571274:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4846 00000100:00100000:0.0:1789760743.571284:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724772165245 time 724822037679 state 0x2 ref 0 00000100:00100000:2.0:1789760743.575804:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.575852:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248000 00000100:00100000:3.0:1789760743.575866:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248000 00000100:00100000:3.0:1789760743.575946:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724772165245 time 724822037679 state 0x0 ref 1 00000100:00100000:3.0:1789760743.575966:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724822298095 time 724822037679 state 0x1 ref 1 00000100:00100000:3.0:1789760743.576004:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724822298095 time 724891307189 state 0x0 ref 1 00000100:00100000:3.0:1789760743.576018:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507248000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760743.576083:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19730432, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760743.576094:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.576114:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760743.576120:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.576128:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.576145:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.616458:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972137 is committed 00002000:00010000:3.0:1789760743.616650:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19730432 -> 19734528 00010000:00010000:3.0:1789760743.616672:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616788:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616800:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616865:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760743.616873:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616886:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616911:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.616920:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b7ff2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.617089:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507248000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41065us (41314us total) trans 4294972137 rc 0/0 00000100:00100000:3.0:1789760743.617102:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4847 00000100:00100000:3.0:1789760743.617114:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724822298095 time 724891307189 state 0x2 ref 0 00000100:00100000:2.0:1789760743.621197:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.621374:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248128 00000100:00100000:0.0:1789760743.621389:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248128 00000100:00100000:0.0:1789760743.621468:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724822298095 time 724891307189 state 0x0 ref 1 00000100:00100000:0.0:1789760743.621488:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724891567605 time 724891307189 state 0x1 ref 1 00000100:00100000:0.0:1789760743.621587:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724891567605 time 724936888377 state 0x0 ref 1 00000100:00100000:0.0:1789760743.621603:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507248128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760743.621675:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19734528, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:0.0:1789760743.621688:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.621740:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760743.621745:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.621753:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.621785:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.645498:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104737100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538273792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645601:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645660:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104736680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538274304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645717:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645746:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104734700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538274816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645786:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645809:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538275328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645833:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645860:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789760743.645868:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538273920:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645894:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645909:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104737480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538274432:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645923:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645934:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104737b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538274944:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645947:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.645974:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538274048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.645994:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.646006:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104736a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538274560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646033:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.646057:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104734380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538275072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646081:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.646105:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760743.646112:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104734000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538274176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646140:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.646166:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538274688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646191:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.646209:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104734a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538275200:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646227:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760743.646266:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274816 00000100:00100000:3.0:1789760743.646266:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273792 00000100:00100000:2.0:1789760743.646275:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274816 00000100:00100000:3.0:1789760743.646275:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273792 00000100:00100000:2.0:1789760743.646326:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275328 00000100:00100000:2.0:1789760743.646329:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275328 00000100:00100000:3.0:1789760743.646331:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789760743.646339:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223494450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538273792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.646344:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274688 00000100:00100000:2.0:1789760743.646346:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274688 00000100:00100000:3.0:1789760743.646401:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223494450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538273792:12345-0@lo:400:kworker.0 Request processed in 61us (800us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.646410:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789760743.646456:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274304 00000100:00100000:3.0:1789760743.646460:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274304 00000100:00100000:3.0:1789760743.646484:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274048 00000100:00100000:3.0:1789760743.646486:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274048 00000100:00100000:3.0:1789760743.646501:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1789760743.646507:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538274304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.646533:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275200 00000100:00100000:2.0:1789760743.646537:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275200 00000100:00100000:3.0:1789760743.646543:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538274304:12345-0@lo:400:kworker.0 Request processed in 36us (851us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.646549:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:2.0:1789760743.646554:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1789760743.646561:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538274816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.646562:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1789760743.646568:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538274048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.646601:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538274816:12345-0@lo:400:kworker.0 Request processed in 41us (814us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760743.646609:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1789760743.646629:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:2.0:1789760743.646636:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538275328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.647216:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538274048:12345-0@lo:400:kworker.0 Request processed in 645us (1217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.647224:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1789760743.647251:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789760743.647260:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538274688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.647306:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538274688:12345-0@lo:400:kworker.0 Request processed in 48us (1114us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.647313:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789760743.647330:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789760743.647337:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538275200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.647366:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538275200:12345-0@lo:400:kworker.0 Request processed in 30us (1137us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.647371:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789760743.647395:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538273920 00000100:00100000:3.0:1789760743.647401:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538273920 00000100:00080000:2.0:1789760743.648005:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760743.648074:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760743.648111:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538275328:12345-0@lo:400:kworker.0 Request processed in 1475us (2276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760743.648119:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:3.0:1789760743.648120:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274432 00000100:00100000:3.0:1789760743.648125:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274432 00000100:00100000:3.0:1789760743.648149:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274944 00000100:00100000:3.0:1789760743.648152:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274944 00000100:00100000:3.0:1789760743.648165:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274560 00000100:00100000:3.0:1789760743.648168:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274560 00000100:00100000:3.0:1789760743.648181:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275072 00000100:00100000:3.0:1789760743.648184:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275072 00000100:00100000:3.0:1789760743.648195:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538274176 00000100:00100000:3.0:1789760743.648198:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538274176 00000100:00100000:3.0:1789760743.648212:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 892 00000100:00100000:3.0:1789760743.648219:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538273920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648261:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538273920:12345-0@lo:400:kworker.0 Request processed in 43us (2365us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648268:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 892 00000100:00100000:3.0:1789760743.648280:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 893 00000100:00100000:3.0:1789760743.648286:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538274432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648321:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538274432:12345-0@lo:400:kworker.0 Request processed in 35us (2395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648329:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 893 00000100:00100000:3.0:1789760743.648342:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1789760743.648346:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227869f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538274944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648371:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227869f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538274944:12345-0@lo:400:kworker.0 Request processed in 25us (2422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648377:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1789760743.648389:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 895 00000100:00100000:3.0:1789760743.648396:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227869180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538274560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648507:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227869180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538274560:12345-0@lo:400:kworker.0 Request processed in 111us (2473us total) trans 0 rc 0/0 00000100:00080000:0.0:1789760743.648511:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760743.648513:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 895 00000100:00080000:0.0:1789760743.648515:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760743.648526:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 896 00000100:00100000:3.0:1789760743.648533:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538275072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648566:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538275072:12345-0@lo:400:kworker.0 Request processed in 34us (2484us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648571:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 896 00000100:00100000:3.0:1789760743.648582:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 897 00000100:00100000:3.0:1789760743.648588:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538274176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.648615:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538274176:12345-0@lo:400:kworker.0 Request processed in 27us (2474us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648622:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 897 00000100:00100000:3.0:1789760743.648724:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538275456:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760743.648779:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.648830:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275456 00000100:00100000:3.0:1789760743.648836:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275456 00000100:00100000:3.0:1789760743.648876:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:3.0:1789760743.648885:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538275456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760743.648919:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760743.648980:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538275456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.648987:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1789760743.649226:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538275456:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760743.649274:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760743.649287:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22782b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538275584:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760743.649321:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760743.649375:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104734000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538274176:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.649395:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538274688:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.649408:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104734a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538275200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.649422:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538274048:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.649435:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104736a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538274560:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.649523:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275584 00000100:00100000:0.0:1789760743.649527:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275584 00000100:00100000:0.0:1789760743.649601:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:0.0:1789760743.649609:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538275584:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760743.649624:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760743.649832:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104734380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538275072:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.649905:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538275584:12345-0@lo:13:osp-pre-1-1.0 Request processed in 296us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760743.649913:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789760743.649974:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22782b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538275584:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760743.650104:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538273920:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650119:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104737480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538274432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650132:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104737b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538274944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650147:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104737100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538273792:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650163:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104736680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538274304:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650315:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104734700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538274816:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.650328:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538275328:0@lo:400:kworker.0 00000001:00080000:0.0:1789760743.657981:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972138 is committed 00002000:00010000:3.0:1789760743.658148:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19734528 -> 19738624 00010000:00010000:3.0:1789760743.658170:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658186:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658196:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658204:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760743.658211:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658222:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658787:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658800:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b7ff9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.659212:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507248128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37331us (37750us total) trans 4294972138 rc 0/0 00000100:00100000:3.0:1789760743.659225:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4848 00000100:00100000:3.0:1789760743.659239:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724891567605 time 724936888377 state 0x2 ref 0 00000100:00100000:0.0:1789760743.662246:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.662443:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248256 00000100:00100000:2.0:1789760743.662456:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248256 00000100:00100000:2.0:1789760743.662532:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724891567605 time 724936888377 state 0x0 ref 1 00000100:00100000:2.0:1789760743.662551:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724937148793 time 724936888377 state 0x1 ref 1 00000100:00100000:2.0:1789760743.662587:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724937148793 time 724977890566 state 0x0 ref 1 00000100:00100000:2.0:1789760743.662601:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.662665:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19738624, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.662676:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.662918:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760743.662924:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.662933:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.662952:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.701577:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972139 is committed 00002000:00010000:3.0:1789760743.701860:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19738624 -> 19742720 00010000:00010000:3.0:1789760743.701889:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701904:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701913:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701920:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760743.701925:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701940:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701966:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.701975:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8000 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.702097:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39491us (39859us total) trans 4294972139 rc 0/0 00000100:00100000:3.0:1789760743.702110:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4849 00000100:00100000:3.0:1789760743.702122:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724937148793 time 724977890566 state 0x2 ref 0 00000100:00100000:2.0:1789760743.712030:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.712188:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248384 00000100:00100000:3.0:1789760743.712202:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248384 00000100:00100000:3.0:1789760743.712275:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724937148793 time 724977890566 state 0x0 ref 1 00000100:00100000:3.0:1789760743.712295:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724978150982 time 724977890566 state 0x1 ref 1 00000100:00100000:3.0:1789760743.712334:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724978150982 time 725027636863 state 0x0 ref 1 00000100:00100000:3.0:1789760743.712349:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760743.712416:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19742720, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760743.712428:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.712450:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760743.712455:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.712463:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.712480:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.743204:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972140 is committed 00002000:00010000:2.0:1789760743.743338:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19742720 -> 19746816 00010000:00010000:2.0:1789760743.743358:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743374:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743384:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743393:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.743398:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743408:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743440:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.743449:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8007 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.743569:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31215us (31547us total) trans 4294972140 rc 0/0 00000100:00100000:2.0:1789760743.743581:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4850 00000100:00100000:2.0:1789760743.743593:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724978150982 time 725027636863 state 0x2 ref 0 00000100:00100000:2.0:1789760743.747481:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.747554:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248512 00000100:00100000:3.0:1789760743.747567:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248512 00000100:00100000:3.0:1789760743.747640:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 724978150982 time 725027636863 state 0x0 ref 1 00000100:00100000:3.0:1789760743.747659:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725027897279 time 725027636863 state 0x1 ref 1 00000100:00100000:3.0:1789760743.747793:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725027897279 time 725063095388 state 0x0 ref 1 00000100:00100000:3.0:1789760743.747818:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507248512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760743.747883:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19746816, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760743.747893:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.747911:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760743.747916:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.747923:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.747944:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.783913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972141 is committed 00002000:00010000:2.0:1789760743.784166:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19746816 -> 19750912 00010000:00010000:2.0:1789760743.784188:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785123:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785136:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785145:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.785151:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785168:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785199:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.785208:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b800e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.785355:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507248512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37542us (37880us total) trans 4294972141 rc 0/0 00000100:00100000:2.0:1789760743.785368:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4851 00000100:00100000:2.0:1789760743.785382:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725027897279 time 725063095388 state 0x2 ref 0 00000100:00100000:0.0:1789760743.790053:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760743.790392:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248640 00000100:00100000:3.0:1789760743.790406:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248640 00000100:00100000:3.0:1789760743.790500:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725027897279 time 725063095388 state 0x0 ref 1 00000100:00100000:3.0:1789760743.790520:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725063355804 time 725063095388 state 0x1 ref 1 00000100:00100000:3.0:1789760743.790558:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725063355804 time 725105860810 state 0x0 ref 1 00000100:00100000:3.0:1789760743.790572:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760743.790642:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19750912, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760743.790654:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.790671:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760743.790676:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.790681:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.790728:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.837837:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972142 is committed 00002000:00010000:2.0:1789760743.838020:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19750912 -> 19755008 00010000:00010000:2.0:1789760743.838041:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838056:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838066:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838074:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.838080:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838092:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838116:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.838125:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8015 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.838297:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47720us (48255us total) trans 4294972142 rc 0/0 00000100:00100000:2.0:1789760743.838310:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4852 00000100:00100000:2.0:1789760743.838322:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725063355804 time 725105860810 state 0x2 ref 0 00000100:00100000:0.0:1789760743.845223:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.845311:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248768 00000100:00100000:2.0:1789760743.845325:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248768 00000100:00100000:2.0:1789760743.845402:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725063355804 time 725105860810 state 0x0 ref 1 00000100:00100000:2.0:1789760743.845422:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725106121226 time 725105860810 state 0x1 ref 1 00000100:00100000:2.0:1789760743.845454:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725106121226 time 725160757921 state 0x0 ref 1 00000100:00100000:2.0:1789760743.845468:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.845537:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19755008, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.845548:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.845569:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760743.845574:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.845582:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.845598:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.886636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972143 is committed 00002000:00010000:0.0:1789760743.887154:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19755008 -> 19759104 00010000:00010000:0.0:1789760743.887175:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887191:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887201:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887208:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.887214:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887224:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887250:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.887260:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b801c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.887383:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507248768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41910us (42167us total) trans 4294972143 rc 0/0 00000100:00100000:0.0:1789760743.887398:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4853 00000100:00100000:0.0:1789760743.887410:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725106121226 time 725160757921 state 0x2 ref 0 00000100:00100000:0.0:1789760743.892101:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.895074:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507248896 00000100:00100000:2.0:1789760743.895095:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507248896 00000100:00100000:2.0:1789760743.895181:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725106121226 time 725160757921 state 0x0 ref 1 00000100:00100000:2.0:1789760743.895202:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725161018337 time 725160757921 state 0x1 ref 1 00000100:00100000:2.0:1789760743.895240:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725161018337 time 725210542086 state 0x0 ref 1 00000100:00100000:2.0:1789760743.895256:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507248896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.895325:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19759104, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.895336:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.895358:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760743.895364:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.895373:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.895390:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.929761:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19759104 -> 19763200 00010000:00010000:2.0:1789760743.929798:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929821:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929831:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929839:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760743.929844:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929854:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929879:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.929888:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8023 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.929932:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972144 is committed 00000100:00100000:2.0:1789760743.930016:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507248896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34756us (37922us total) trans 4294972144 rc 0/0 00000100:00100000:2.0:1789760743.930029:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4854 00000100:00100000:2.0:1789760743.930040:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725161018337 time 725210542086 state 0x2 ref 0 00000100:00100000:2.0:1789760743.937297:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760743.937404:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249024 00000100:00100000:0.0:1789760743.937417:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249024 00000100:00100000:0.0:1789760743.937489:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725161018337 time 725210542086 state 0x0 ref 1 00000100:00100000:0.0:1789760743.937509:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725210802502 time 725210542086 state 0x1 ref 1 00000100:00100000:0.0:1789760743.937544:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725210802502 time 725252846501 state 0x0 ref 1 00000100:00100000:0.0:1789760743.937558:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507249024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760743.937619:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19763200, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:0.0:1789760743.937629:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.937649:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760743.937654:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.937661:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.937678:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.968322:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972145 is committed 00002000:00010000:0.0:1789760743.968613:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19763200 -> 19767296 00010000:00010000:0.0:1789760743.968634:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968649:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968659:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968666:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760743.968672:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968682:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968724:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.968732:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b802a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.969029:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507249024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31464us (31739us total) trans 4294972145 rc 0/0 00000100:00100000:0.0:1789760743.969045:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4855 00000100:00100000:0.0:1789760743.969058:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725210802502 time 725252846501 state 0x2 ref 0 00000100:00100000:0.0:1789760743.975850:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760743.976240:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249152 00000100:00100000:2.0:1789760743.976253:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249152 00000100:00100000:2.0:1789760743.976334:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725210802502 time 725252846501 state 0x0 ref 1 00000100:00100000:2.0:1789760743.976354:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725253106917 time 725252846501 state 0x1 ref 1 00000100:00100000:2.0:1789760743.976438:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725253106917 time 725291739372 state 0x0 ref 1 00000100:00100000:2.0:1789760743.976454:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760743.976524:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19767296, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:2.0:1789760743.976536:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.976557:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760743.976562:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.976570:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.976588:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.013544:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972146 is committed 00002000:00010000:0.0:1789760744.013649:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19767296 -> 19771392 00010000:00010000:0.0:1789760744.013669:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013684:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013723:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013732:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.013737:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013747:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013783:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.013796:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8031 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.013929:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37471us (38088us total) trans 4294972146 rc 0/0 00000100:00100000:0.0:1789760744.013941:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4856 00000100:00100000:0.0:1789760744.013957:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725253106917 time 725291739372 state 0x2 ref 0 00000100:00100000:2.0:1789760744.024971:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760744.025386:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249280 00000100:00100000:3.0:1789760744.025401:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249280 00000100:00100000:3.0:1789760744.025476:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725253106917 time 725291739372 state 0x0 ref 1 00000100:00100000:3.0:1789760744.025496:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725291999788 time 725291739372 state 0x1 ref 1 00000100:00100000:3.0:1789760744.025600:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725291999788 time 725340901808 state 0x0 ref 1 00000100:00100000:3.0:1789760744.025617:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507249280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760744.025685:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19771392, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x9807 00010000:00010000:3.0:1789760744.025744:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.025766:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760744.025781:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.025789:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.025815:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.065439:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972147 is committed 00002000:00010000:3.0:1789760744.065656:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19771392 -> 19775488 00002000:00010000:3.0:1789760744.065662:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:3.0:1789760744.065665:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:3.0:1789760744.065681:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065724:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065736:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065744:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760744.065749:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065759:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065794:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.065803:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8038 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.066081:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507249280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40461us (41118us total) trans 4294972147 rc 0/0 00000100:00100000:3.0:1789760744.066094:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4857 00000100:00100000:3.0:1789760744.066106:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725291999788 time 725340901808 state 0x2 ref 0 00000100:00100000:2.0:1789760744.069470:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.069597:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249408 00000100:00100000:0.0:1789760744.069610:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249408 00000100:00100000:0.0:1789760744.069684:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725291999788 time 725340901808 state 0x0 ref 1 00000100:00100000:0.0:1789760744.069744:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725341162224 time 725340901808 state 0x1 ref 1 00000100:00100000:0.0:1789760744.069779:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725341162224 time 725385072327 state 0x0 ref 1 00000100:00100000:0.0:1789760744.069794:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.069869:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19775488, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.069880:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.069900:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.069905:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.069914:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.069932:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.112081:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972148 is committed 00002000:00010000:0.0:1789760744.112187:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19775488 -> 19779584 00010000:00010000:0.0:1789760744.112206:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112296:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112306:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112314:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.112320:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112331:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112357:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.112365:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b803f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.112530:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42733us (43070us total) trans 4294972148 rc 0/0 00000100:00100000:0.0:1789760744.112548:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4858 00000100:00100000:0.0:1789760744.112561:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725341162224 time 725385072327 state 0x2 ref 0 00000100:00100000:0.0:1789760744.115926:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.116031:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249536 00000100:00100000:2.0:1789760744.116044:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249536 00000100:00100000:2.0:1789760744.116121:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725341162224 time 725385072327 state 0x0 ref 1 00000100:00100000:2.0:1789760744.116140:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725385332743 time 725385072327 state 0x1 ref 1 00000100:00100000:2.0:1789760744.116165:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725385332743 time 725431468555 state 0x0 ref 1 00000100:00100000:2.0:1789760744.116179:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507249536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.116246:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19779584, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.116257:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.116278:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760744.116284:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.116293:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.116311:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.162405:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19779584 -> 19783680 00000001:00080000:1.0:1789760744.162433:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972149 is committed 00010000:00010000:0.0:1789760744.162436:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162450:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162460:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162468:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.162474:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162484:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162507:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.162516:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8046 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.162644:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507249536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46460us (46726us total) trans 4294972149 rc 0/0 00000100:00100000:0.0:1789760744.162657:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4859 00000100:00100000:0.0:1789760744.162670:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725385332743 time 725431468555 state 0x2 ref 0 00000100:00100000:2.0:1789760744.165659:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760744.165787:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249664 00000100:00100000:3.0:1789760744.165800:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249664 00000100:00100000:3.0:1789760744.165874:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725385332743 time 725431468555 state 0x0 ref 1 00000100:00100000:3.0:1789760744.165893:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725431728971 time 725431468555 state 0x1 ref 1 00000100:00100000:3.0:1789760744.165928:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725431728971 time 725481231564 state 0x0 ref 1 00000100:00100000:3.0:1789760744.165941:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760744.166007:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19783680, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:3.0:1789760744.166018:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.166038:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760744.166043:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.166051:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.166067:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.205566:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972150 is committed 00002000:00010000:0.0:1789760744.205720:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19783680 -> 19787776 00010000:00010000:0.0:1789760744.205741:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.205755:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.205765:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.205785:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.205791:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.205802:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.206310:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.206325:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b804d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.206510:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40561us (40856us total) trans 4294972150 rc 0/0 00000100:00100000:0.0:1789760744.206523:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4860 00000100:00100000:0.0:1789760744.206536:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725431728971 time 725481231564 state 0x2 ref 0 00000100:00100000:2.0:1789760744.210795:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.210897:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249792 00000100:00100000:0.0:1789760744.210910:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249792 00000100:00100000:0.0:1789760744.210993:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725431728971 time 725481231564 state 0x0 ref 1 00000100:00100000:0.0:1789760744.211014:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725481491980 time 725481231564 state 0x1 ref 1 00000100:00100000:0.0:1789760744.211051:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725481491980 time 725526353720 state 0x0 ref 1 00000100:00100000:0.0:1789760744.211066:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.211132:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19787776, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.211142:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.211161:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.211167:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.211174:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.211197:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.247991:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972151 is committed 00002000:00010000:1.0:1789760744.248109:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19787776 -> 19791872 00010000:00010000:1.0:1789760744.248129:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248145:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248154:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248160:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760744.248165:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248434:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248462:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.248471:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8054 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.248744:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507249792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37629us (37923us total) trans 4294972151 rc 0/0 00000100:00100000:1.0:1789760744.248758:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4861 00000100:00100000:1.0:1789760744.248781:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725481491980 time 725526353720 state 0x2 ref 0 00000100:00100000:2.0:1789760744.253300:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.253449:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507249920 00000100:00100000:0.0:1789760744.253463:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507249920 00000100:00100000:0.0:1789760744.253542:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725481491980 time 725526353720 state 0x0 ref 1 00000100:00100000:0.0:1789760744.253560:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725526614136 time 725526353720 state 0x1 ref 1 00000100:00100000:0.0:1789760744.253597:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725526614136 time 725568900661 state 0x0 ref 1 00000100:00100000:0.0:1789760744.253609:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507249920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.253676:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19791872, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.253686:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.253734:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.253740:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.253748:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.253776:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.289639:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19791872 -> 19795968 00010000:00010000:0.0:1789760744.289670:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289687:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289720:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289729:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.289735:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289745:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289785:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.289794:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b805b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.289936:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507249920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36320us (36645us total) trans 4294972152 rc 0/0 00000100:00100000:0.0:1789760744.289948:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4862 00000100:00100000:0.0:1789760744.289961:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725526614136 time 725568900661 state 0x2 ref 0 00000001:00080000:1.0:1789760744.290952:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972152 is committed 00000100:00100000:2.0:1789760744.296833:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760744.297293:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250048 00000100:00100000:3.0:1789760744.297307:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250048 00000100:00100000:3.0:1789760744.297383:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725526614136 time 725568900661 state 0x0 ref 1 00000100:00100000:3.0:1789760744.297402:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725569161077 time 725568900661 state 0x1 ref 1 00000100:00100000:3.0:1789760744.297442:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725569161077 time 725612745308 state 0x0 ref 1 00000100:00100000:3.0:1789760744.297458:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760744.297525:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19795968, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:3.0:1789760744.297539:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.297558:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760744.297564:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.297572:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.297589:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.348184:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760744.354521:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972153 is committed 00002000:00010000:0.0:1789760744.354866:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19795968 -> 19800064 00010000:00010000:0.0:1789760744.354888:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.354905:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.354916:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.354924:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.354930:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.355188:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.355214:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.355223:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8062 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.355342:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57881us (58518us total) trans 4294972153 rc 0/0 00000100:00100000:0.0:1789760744.355354:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4863 00000100:00100000:0.0:1789760744.355367:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725569161077 time 725612745308 state 0x2 ref 0 00000100:00100000:3.0:1789760744.359422:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760744.359589:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250176 00000100:00100000:1.0:1789760744.359603:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250176 00000100:00100000:1.0:1789760744.359901:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725569161077 time 725612745308 state 0x0 ref 1 00000100:00100000:1.0:1789760744.359925:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725613005724 time 725612745308 state 0x1 ref 1 00000100:00100000:0.0:1789760744.360169:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725613005724 time 725675469532 state 0x0 ref 1 00000100:00100000:0.0:1789760744.360191:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507250176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.360261:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19800064, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.360272:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.360292:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.360298:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.360306:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.360324:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.398319:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972154 is committed 00002000:00010000:0.0:1789760744.398592:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19800064 -> 19804160 00010000:00010000:0.0:1789760744.398612:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398627:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398637:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398650:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.398657:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398667:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398689:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.398712:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8069 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.398934:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507250176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38744us (39521us total) trans 4294972154 rc 0/0 00000100:00100000:0.0:1789760744.398946:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4864 00000100:00100000:0.0:1789760744.398958:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725613005724 time 725675469532 state 0x2 ref 0 00000100:00100000:2.0:1789760744.404107:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.404206:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250304 00000100:00100000:0.0:1789760744.404231:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250304 00000100:00100000:0.0:1789760744.404307:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725613005724 time 725675469532 state 0x0 ref 1 00000100:00100000:0.0:1789760744.404327:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725675729948 time 725675469532 state 0x1 ref 1 00000100:00100000:0.0:1789760744.404360:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725675729948 time 725719663185 state 0x0 ref 1 00000100:00100000:0.0:1789760744.404379:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.404445:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19804160, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.404456:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.404477:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.404482:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.404490:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.404508:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.452600:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972155 is committed 00002000:00010000:0.0:1789760744.452780:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19804160 -> 19808256 00010000:00010000:0.0:1789760744.452925:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.452943:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.452955:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.452965:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.452972:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.452984:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.453010:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.453017:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8070 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.453198:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48815us (49098us total) trans 4294972155 rc 0/0 00000100:00100000:0.0:1789760744.453222:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4865 00000100:00100000:0.0:1789760744.453235:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725675729948 time 725719663185 state 0x2 ref 0 00000100:00100000:2.0:1789760744.456353:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.456447:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250432 00000100:00100000:0.0:1789760744.456459:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250432 00000100:00100000:0.0:1789760744.456540:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725675729948 time 725719663185 state 0x0 ref 1 00000100:00100000:0.0:1789760744.456560:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725719923601 time 725719663185 state 0x1 ref 1 00000100:00100000:0.0:1789760744.456594:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725719923601 time 725771897053 state 0x0 ref 1 00000100:00100000:0.0:1789760744.456609:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507250432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.456675:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19808256, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.456687:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.456741:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.456747:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.456755:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.456785:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760744.498373:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19808256 -> 19812352 00010000:00010000:2.0:1789760744.498403:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498418:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498427:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498435:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760744.498440:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498449:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498479:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.498486:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8077 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.498625:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507250432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42010us (42277us total) trans 4294972156 rc 0/0 00000100:00100000:2.0:1789760744.498638:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4866 00000100:00100000:2.0:1789760744.498651:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725719923601 time 725771897053 state 0x2 ref 0 00000001:00080000:1.0:1789760744.499657:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972156 is committed 00000100:00100000:3.0:1789760744.506099:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.506373:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250560 00000100:00100000:0.0:1789760744.506388:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250560 00000100:00100000:0.0:1789760744.506462:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725719923601 time 725771897053 state 0x0 ref 1 00000100:00100000:0.0:1789760744.506484:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725772157469 time 725771897053 state 0x1 ref 1 00000100:00100000:0.0:1789760744.506577:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725772157469 time 725821878774 state 0x0 ref 1 00000100:00100000:0.0:1789760744.506592:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760744.506660:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19812352, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760744.506670:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.506690:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760744.506746:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.506754:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.506788:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.542016:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972157 is committed 00002000:00010000:2.0:1789760744.542240:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19812352 -> 19816448 00010000:00010000:2.0:1789760744.542260:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542276:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542287:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542295:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760744.542300:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542311:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542336:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.542349:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b807e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.542476:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507250560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35875us (36382us total) trans 4294972157 rc 0/0 00000100:00100000:2.0:1789760744.542488:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4867 00000100:00100000:2.0:1789760744.542501:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725772157469 time 725821878774 state 0x2 ref 0 00000100:00100000:0.0:1789760744.550367:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.550515:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250688 00000100:00100000:2.0:1789760744.550528:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250688 00000100:00100000:2.0:1789760744.550677:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725772157469 time 725821878774 state 0x0 ref 1 00000100:00100000:2.0:1789760744.550934:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725822139190 time 725821878774 state 0x1 ref 1 00000100:00100000:1.0:1789760744.551034:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725822139190 time 725866333877 state 0x0 ref 1 00000100:00100000:1.0:1789760744.551052:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507250688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760744.551126:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19816448, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:1.0:1789760744.551137:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.551159:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760744.551165:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.551172:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.551190:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.583638:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19816448 -> 19820544 00010000:00010000:0.0:1789760744.583672:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583711:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583724:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583745:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.583751:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583763:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583797:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.583814:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8085 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.583950:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507250688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32895us (33590us total) trans 4294972158 rc 0/0 00000100:00100000:0.0:1789760744.583963:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4868 00000100:00100000:0.0:1789760744.583976:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725822139190 time 725866333877 state 0x2 ref 0 00000001:00080000:2.0:1789760744.584508:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972158 is committed 00000100:00100000:0.0:1789760744.589032:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760744.589483:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250816 00000100:00100000:3.0:1789760744.589499:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250816 00000100:00100000:3.0:1789760744.589589:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725822139190 time 725866333877 state 0x0 ref 1 00000100:00100000:3.0:1789760744.589609:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725866594293 time 725866333877 state 0x1 ref 1 00000100:00100000:3.0:1789760744.589740:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725866594293 time 725905031461 state 0x0 ref 1 00000100:00100000:3.0:1789760744.589776:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507250816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760744.589868:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19820544, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:3.0:1789760744.589881:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.589909:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760744.589915:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.589923:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.589941:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.629455:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972159 is committed 00002000:00010000:2.0:1789760744.629575:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19820544 -> 19824640 00010000:00010000:2.0:1789760744.629595:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629609:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629619:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629627:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760744.629633:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629644:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629670:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629678:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b808c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.629915:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507250816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40152us (40892us total) trans 4294972159 rc 0/0 00000100:00100000:2.0:1789760744.629928:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4869 00000100:00100000:2.0:1789760744.629941:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725866594293 time 725905031461 state 0x2 ref 0 00000100:00100000:1.0:1789760744.636397:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.636869:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507250944 00000100:00100000:2.0:1789760744.636884:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507250944 00000100:00100000:2.0:1789760744.636959:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725866594293 time 725905031461 state 0x0 ref 1 00000100:00100000:2.0:1789760744.636978:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725905291877 time 725905031461 state 0x1 ref 1 00000100:00100000:2.0:1789760744.637005:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725905291877 time 725952308891 state 0x0 ref 1 00000100:00100000:2.0:1789760744.637020:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507250944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.637392:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19824640, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.637406:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.637427:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760744.637433:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.637441:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.637458:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.668901:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760744.668935:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fe9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538275712:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760744.669041:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760744.669131:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275712 00000100:00100000:3.0:1789760744.669141:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275712 00000100:00100000:1.0:1789760744.669541:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:1.0:1789760744.669556:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538275712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760744.669587:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3838836736 avail 3794796544 00000100:00100000:1.0:1789760744.669654:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538275712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760744.669662:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:1.0:1789760744.669995:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538275840:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760744.670038:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760744.670076:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275840 00000100:00100000:0.0:1789760744.670084:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275840 00000100:00100000:0.0:1789760744.670130:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1789760744.670138:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538275840:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760744.670154:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3838836736 avail 3794796544 00000100:00100000:0.0:1789760744.671446:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538275840:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1302us (1400us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760744.671459:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1789760744.671537:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538275840:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760744.671886:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fe9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538275712:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760744.686948:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972160 is committed 00002000:00010000:1.0:1789760744.687141:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19824640 -> 19828736 00010000:00010000:1.0:1789760744.687163:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687185:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687812:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687824:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760744.687830:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687839:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687873:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.687882:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8093 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.688004:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507250944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50979us (51612us total) trans 4294972160 rc 0/0 00000100:00100000:1.0:1789760744.688017:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4870 00000100:00100000:1.0:1789760744.688028:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725905291877 time 725952308891 state 0x2 ref 0 00000100:00100000:0.0:1789760744.691607:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760744.691680:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251072 00000100:00100000:1.0:1789760744.691746:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251072 00000100:00100000:1.0:1789760744.692072:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725905291877 time 725952308891 state 0x0 ref 1 00000100:00100000:1.0:1789760744.692093:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725952569307 time 725952308891 state 0x1 ref 1 00000100:00100000:1.0:1789760744.692132:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725952569307 time 726007434906 state 0x0 ref 1 00000100:00100000:1.0:1789760744.692147:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21face050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507251072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760744.692214:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19828736, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:1.0:1789760744.692226:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.692246:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760744.692251:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.692259:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.692275:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.733549:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972161 is committed 00002000:00010000:0.0:1789760744.733688:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19828736 -> 19832832 00010000:00010000:0.0:1789760744.733740:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733757:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733784:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733792:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.733798:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733874:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733899:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.733907:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b809a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.734204:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21face050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507251072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42053us (42604us total) trans 4294972161 rc 0/0 00000100:00100000:0.0:1789760744.734217:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4871 00000100:00100000:0.0:1789760744.734231:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725952569307 time 726007434906 state 0x2 ref 0 00000100:00100000:1.0:1789760744.743683:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.743863:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251200 00000100:00100000:2.0:1789760744.743877:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251200 00000100:00100000:2.0:1789760744.743965:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 725952569307 time 726007434906 state 0x0 ref 1 00000100:00100000:2.0:1789760744.743993:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726007695322 time 726007434906 state 0x1 ref 1 00000100:00100000:2.0:1789760744.744021:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726007695322 time 726059323937 state 0x0 ref 1 00000100:00100000:2.0:1789760744.744036:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507251200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.744113:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19832832, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.744124:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.744148:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760744.744154:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.744162:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.744180:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.802304:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19832832 -> 19836928 00010000:00010000:0.0:1789760744.802342:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802364:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802374:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802382:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.802388:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802400:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802427:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.802435:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b80a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.802572:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507251200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58531us (58897us total) trans 4294972162 rc 0/0 00000100:00100000:0.0:1789760744.802585:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4872 00000100:00100000:0.0:1789760744.802598:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726007695322 time 726059323937 state 0x2 ref 0 00000001:00080000:2.0:1789760744.805080:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972162 is committed 00000100:00100000:1.0:1789760744.805215:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.805306:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251328 00000100:00100000:2.0:1789760744.805318:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251328 00000100:00100000:2.0:1789760744.805462:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726007695322 time 726059323937 state 0x0 ref 1 00000100:00100000:2.0:1789760744.805483:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726059584353 time 726059323937 state 0x1 ref 1 00000100:00100000:2.0:1789760744.805564:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726059584353 time 726120865657 state 0x0 ref 1 00000100:00100000:2.0:1789760744.805579:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507251328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.805644:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19836928, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.805655:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.805674:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760744.805679:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.805686:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.805731:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.858418:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972163 is committed 00002000:00010000:1.0:1789760744.859148:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19836928 -> 19841024 00010000:00010000:1.0:1789760744.859177:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859192:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859203:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859211:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760744.859217:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859227:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859323:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.859333:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b80a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.859578:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507251328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53996us (54364us total) trans 4294972163 rc 0/0 00000100:00100000:1.0:1789760744.859587:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4873 00000100:00100000:1.0:1789760744.859597:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726059584353 time 726120865657 state 0x2 ref 0 00000100:00100000:1.0:1789760744.868288:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760744.869660:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251456 00000100:00100000:0.0:1789760744.869688:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251456 00000100:00100000:2.0:1789760744.873274:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726059584353 time 726120865657 state 0x0 ref 1 00000100:00100000:2.0:1789760744.873302:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726121126073 time 726120865657 state 0x1 ref 1 00000100:00100000:2.0:1789760744.873455:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726121126073 time 726188755959 state 0x0 ref 1 00000100:00100000:2.0:1789760744.873475:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.873549:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19841024, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.873561:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.873583:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760744.873588:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.873595:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.873613:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.921527:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972164 is committed 00002000:00010000:0.0:1789760744.921713:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19841024 -> 19845120 00010000:00010000:0.0:1789760744.921734:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921753:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921774:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921783:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760744.921789:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921809:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921837:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.921846:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b80af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.921977:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48501us (53697us total) trans 4294972164 rc 0/0 00000100:00100000:0.0:1789760744.921992:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4874 00000100:00100000:0.0:1789760744.922006:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726121126073 time 726188755959 state 0x2 ref 0 00000100:00100000:0.0:1789760744.930292:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760744.930809:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251584 00000100:00100000:2.0:1789760744.930824:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251584 00000100:00100000:2.0:1789760744.930919:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726121126073 time 726188755959 state 0x0 ref 1 00000100:00100000:2.0:1789760744.930945:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726189016375 time 726188755959 state 0x1 ref 1 00000100:00100000:2.0:1789760744.930978:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726189016375 time 726246280867 state 0x0 ref 1 00000100:00100000:2.0:1789760744.930994:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507251584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760744.931058:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19845120, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:2.0:1789760744.931070:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.931091:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760744.931096:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.931103:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.931121:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.978246:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972165 is committed 00002000:00010000:2.0:1789760744.978321:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19845120 -> 19849216 00010000:00010000:2.0:1789760744.978343:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978358:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978368:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978382:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760744.978388:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978399:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978425:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.978433:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.978578:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507251584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47579us (48296us total) trans 4294972165 rc 0/0 00000100:00100000:2.0:1789760744.979727:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4875 00000100:00100000:2.0:1789760744.979745:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726189016375 time 726246280867 state 0x2 ref 0 00000100:00100000:3.0:1789760744.984146:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760744.984629:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251712 00000100:00100000:1.0:1789760744.984641:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251712 00000100:00100000:1.0:1789760744.984750:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726189016375 time 726246280867 state 0x0 ref 1 00000100:00100000:1.0:1789760744.984784:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726246541283 time 726246280867 state 0x1 ref 1 00000100:00100000:1.0:1789760744.984825:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726246541283 time 726300127825 state 0x0 ref 1 00000100:00100000:1.0:1789760744.984839:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760744.984906:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19849216, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:1.0:1789760744.984917:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.984938:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760744.984943:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.984951:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.984967:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.016842:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972166 is committed 00002000:00010000:2.0:1789760745.017074:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19849216 -> 19853312 00010000:00010000:2.0:1789760745.017094:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017109:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017123:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017131:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760745.017136:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017145:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017169:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.017177:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b80bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.017298:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32453us (33159us total) trans 4294972166 rc 0/0 00000100:00100000:2.0:1789760745.017311:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4876 00000100:00100000:2.0:1789760745.017620:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726246541283 time 726300127825 state 0x2 ref 0 00000100:00100000:1.0:1789760745.025682:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760745.026552:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251840 00000100:00100000:0.0:1789760745.026569:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251840 00000100:00100000:0.0:1789760745.026740:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726246541283 time 726300127825 state 0x0 ref 1 00000100:00100000:0.0:1789760745.026758:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726300388241 time 726300127825 state 0x1 ref 1 00000100:00100000:0.0:1789760745.026812:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726300388241 time 726342115077 state 0x0 ref 1 00000100:00100000:0.0:1789760745.026830:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760745.026893:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19853312, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x9807 00010000:00010000:0.0:1789760745.026906:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026925:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760745.026931:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.026938:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026956:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760745.030903:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760745.070019:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19853312 -> 19857408 00002000:00010000:0.0:1789760745.070219:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:0.0:1789760745.070222:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:0.0:1789760745.070240:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070408:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070418:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070427:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760745.070433:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070443:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070470:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.070478:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.070891:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972167 is committed 00000100:00100000:1.0:1789760745.071485:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44651us (45808us total) trans 4294972167 rc 0/0 00000100:00100000:1.0:1789760745.071500:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4877 00000100:00100000:1.0:1789760745.071514:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726300388241 time 726342115077 state 0x2 ref 0 00000100:00100000:1.0:1789760745.075584:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760745.075732:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507251968 00000100:00100000:2.0:1789760745.075746:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507251968 00000100:00100000:2.0:1789760745.075948:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726300388241 time 726342115077 state 0x0 ref 1 00000100:00100000:2.0:1789760745.075970:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726342375493 time 726342115077 state 0x1 ref 1 00000100:00100000:2.0:1789760745.076003:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726342375493 time 726391305345 state 0x0 ref 1 00000100:00100000:2.0:1789760745.076018:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760745.076087:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19857408, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:2.0:1789760745.076098:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.076118:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760745.076123:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.076132:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.076148:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.118359:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972168 is committed 00002000:00010000:0.0:1789760745.118606:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19857408 -> 19861504 00010000:00010000:0.0:1789760745.118628:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118649:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118659:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118667:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760745.118673:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118684:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118729:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.118738:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b80cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.118933:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507251968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42910us (43357us total) trans 4294972168 rc 0/0 00000100:00100000:0.0:1789760745.118946:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4878 00000100:00100000:0.0:1789760745.118958:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726342375493 time 726391305345 state 0x2 ref 0 00000100:00100000:3.0:1789760745.122350:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760745.122478:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252096 00000100:00100000:2.0:1789760745.122492:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252096 00000100:00100000:2.0:1789760745.122581:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726342375493 time 726391305345 state 0x0 ref 1 00000100:00100000:2.0:1789760745.122601:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726391565761 time 726391305345 state 0x1 ref 1 00000100:00100000:2.0:1789760745.122729:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726391565761 time 726438031210 state 0x0 ref 1 00000100:00100000:2.0:1789760745.122745:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760745.122825:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19861504, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:2.0:1789760745.122836:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.122856:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760745.122862:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.122870:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.122888:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.159018:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972169 is committed 00002000:00010000:3.0:1789760745.159302:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19861504 -> 19865600 00010000:00010000:3.0:1789760745.159323:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159339:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159350:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159364:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.159370:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159381:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159405:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.159414:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b80d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.159542:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36791us (37199us total) trans 4294972169 rc 0/0 00000100:00100000:3.0:1789760745.159555:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4879 00000100:00100000:3.0:1789760745.159567:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726391565761 time 726438031210 state 0x2 ref 0 00000100:00100000:1.0:1789760745.170499:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760745.170626:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252224 00000100:00100000:2.0:1789760745.170640:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252224 00000100:00100000:2.0:1789760745.170890:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726391565761 time 726438031210 state 0x0 ref 1 00000100:00100000:2.0:1789760745.170909:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726438291626 time 726438031210 state 0x1 ref 1 00000100:00100000:3.0:1789760745.171959:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726438291626 time 726487251416 state 0x0 ref 1 00000100:00100000:3.0:1789760745.171989:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507252224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.172062:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19865600, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.172073:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.172099:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.172105:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.172112:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.172129:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.184337:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538275968:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760745.184440:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760745.184543:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538275968 00000100:00100000:0.0:1789760745.184553:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538275968 00000100:00100000:0.0:1789760745.184612:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789760745.184622:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538275968:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760745.184674:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760745.184749:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538275968:12345-0@lo:41:osp-pre-1-0.0 Request processed in 127us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760745.184756:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789760745.184827:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538275968:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760745.221886:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972170 is committed 00002000:00010000:1.0:1789760745.222087:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19865600 -> 19869696 00010000:00010000:1.0:1789760745.222106:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222122:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222132:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222141:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.222147:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222157:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222181:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.222189:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b80d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.222414:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507252224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50429us (51923us total) trans 4294972170 rc 0/0 00000100:00100000:1.0:1789760745.222427:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4880 00000100:00100000:1.0:1789760745.222440:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726438291626 time 726487251416 state 0x2 ref 0 00000100:00100000:3.0:1789760745.225805:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760745.226011:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252352 00000100:00100000:0.0:1789760745.226024:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252352 00000100:00100000:0.0:1789760745.226091:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726438291626 time 726487251416 state 0x0 ref 1 00000100:00100000:0.0:1789760745.226109:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726487511832 time 726487251416 state 0x1 ref 1 00000100:00100000:0.0:1789760745.226163:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726487511832 time 726541465685 state 0x0 ref 1 00000100:00100000:0.0:1789760745.226176:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760745.226247:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19869696, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:0.0:1789760745.226258:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.226277:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760745.226283:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.226290:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.226306:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.270025:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972171 is committed 00002000:00010000:0.0:1789760745.270177:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19869696 -> 19873792 00010000:00010000:0.0:1789760745.270286:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270300:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270310:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270317:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760745.270322:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270331:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270349:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.270356:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.270471:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44292us (44685us total) trans 4294972171 rc 0/0 00000100:00100000:0.0:1789760745.270482:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4881 00000100:00100000:0.0:1789760745.270491:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726487511832 time 726541465685 state 0x2 ref 0 00000100:00100000:1.0:1789760745.277493:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.277601:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252480 00000100:00100000:3.0:1789760745.277610:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252480 00000100:00100000:3.0:1789760745.277662:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726487511832 time 726541465685 state 0x0 ref 1 00000100:00100000:3.0:1789760745.277678:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726541726101 time 726541465685 state 0x1 ref 1 00000100:00100000:3.0:1789760745.277797:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726541726101 time 726593098489 state 0x0 ref 1 00000100:00100000:3.0:1789760745.277810:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507252480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.277855:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19873792, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.277865:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.277881:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.277885:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.277892:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.277905:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.304876:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19873792 -> 19877888 00010000:00010000:3.0:1789760745.304896:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304910:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304921:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304929:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.304934:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304944:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304964:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.304972:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.305066:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507252480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27253us (27577us total) trans 4294972172 rc 0/0 00000100:00100000:3.0:1789760745.305076:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4882 00000100:00100000:3.0:1789760745.305086:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726541726101 time 726593098489 state 0x2 ref 0 00000001:00080000:3.0:1789760745.305134:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972172 is committed 00000100:00100000:2.0:1789760745.314320:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760745.314414:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252608 00000100:00100000:1.0:1789760745.314423:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252608 00000100:00100000:1.0:1789760745.314480:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726541726101 time 726593098489 state 0x0 ref 1 00000100:00100000:1.0:1789760745.314558:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726593358905 time 726593098489 state 0x1 ref 1 00000100:00100000:1.0:1789760745.314592:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726593358905 time 726629895723 state 0x0 ref 1 00000100:00100000:1.0:1789760745.314604:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760745.314652:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19877888, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:1.0:1789760745.314661:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.314676:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760745.314680:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.314686:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.314718:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.365441:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19877888 -> 19881984 00010000:00010000:1.0:1789760745.365462:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365482:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365571:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365579:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.365583:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365593:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365612:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.365620:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b80ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.365749:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507252608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51141us (51432us total) trans 4294972173 rc 0/0 00000100:00100000:1.0:1789760745.365772:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4883 00000100:00100000:1.0:1789760745.365782:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726593358905 time 726629895723 state 0x2 ref 0 00000001:00080000:2.0:1789760745.366299:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972173 is committed 00000100:00100000:1.0:1789760745.368896:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760745.369040:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252736 00000100:00100000:0.0:1789760745.369050:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252736 00000100:00100000:0.0:1789760745.369105:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726593358905 time 726629895723 state 0x0 ref 1 00000100:00100000:0.0:1789760745.369121:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726630156139 time 726629895723 state 0x1 ref 1 00000100:00100000:0.0:1789760745.369157:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726630156139 time 726684459476 state 0x0 ref 1 00000100:00100000:0.0:1789760745.369170:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760745.369215:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19881984, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:0.0:1789760745.369224:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.369238:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760745.369242:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.369248:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.369262:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.372360:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760745.408976:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972174 is committed 00002000:00010000:0.0:1789760745.409143:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19881984 -> 19886080 00010000:00010000:0.0:1789760745.409159:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409172:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409182:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409190:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760745.409195:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409204:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409226:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.409234:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b80f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.409326:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40153us (40433us total) trans 4294972174 rc 0/0 00000100:00100000:0.0:1789760745.409338:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4884 00000100:00100000:0.0:1789760745.409348:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726630156139 time 726684459476 state 0x2 ref 0 00000100:00100000:1.0:1789760745.412998:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.413543:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252864 00000100:00100000:3.0:1789760745.413561:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252864 00000100:00100000:3.0:1789760745.413644:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726630156139 time 726684459476 state 0x0 ref 1 00000100:00100000:3.0:1789760745.413663:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726684719892 time 726684459476 state 0x1 ref 1 00000100:00100000:3.0:1789760745.413769:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726684719892 time 726729062268 state 0x0 ref 1 00000100:00100000:3.0:1789760745.413787:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.413862:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19886080, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.413874:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.413894:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.413900:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.413907:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.413925:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.453880:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19886080 -> 19890176 00010000:00010000:1.0:1789760745.453908:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.453923:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.453933:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.453940:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.453945:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.454293:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.454321:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.454330:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b80fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.454465:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507252864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40674us (41467us total) trans 4294972175 rc 0/0 00000100:00100000:1.0:1789760745.454479:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4885 00000100:00100000:1.0:1789760745.454492:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726684719892 time 726729062268 state 0x2 ref 0 00000001:00080000:2.0:1789760745.455407:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972175 is committed 00000100:00100000:2.0:1789760745.465211:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760745.465296:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507252992 00000100:00100000:1.0:1789760745.465309:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507252992 00000100:00100000:1.0:1789760745.465385:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726684719892 time 726729062268 state 0x0 ref 1 00000100:00100000:1.0:1789760745.465406:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726729322684 time 726729062268 state 0x1 ref 1 00000100:00100000:1.0:1789760745.465442:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726729322684 time 726780744618 state 0x0 ref 1 00000100:00100000:1.0:1789760745.465456:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507252992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760745.466308:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19890176, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:1.0:1789760745.466323:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.466346:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760745.466351:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.466359:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.466378:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.507600:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19890176 -> 19894272 00010000:00010000:3.0:1789760745.507625:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507642:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507652:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507660:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.507665:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507674:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507719:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.507729:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8103 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.507933:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507252992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42471us (42740us total) trans 4294972176 rc 0/0 00000100:00100000:3.0:1789760745.507950:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4886 00000100:00100000:3.0:1789760745.507963:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726729322684 time 726780744618 state 0x2 ref 0 00000001:00080000:2.0:1789760745.508657:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972176 is committed 00000100:00100000:1.0:1789760745.511018:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760745.511090:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253120 00000100:00100000:0.0:1789760745.511101:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253120 00000100:00100000:0.0:1789760745.511167:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726729322684 time 726780744618 state 0x0 ref 1 00000100:00100000:0.0:1789760745.511185:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726781005034 time 726780744618 state 0x1 ref 1 00000100:00100000:0.0:1789760745.511305:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726781005034 time 726826607131 state 0x0 ref 1 00000100:00100000:0.0:1789760745.511319:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760745.511379:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19894272, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:0.0:1789760745.511390:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.511408:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760745.511413:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.511420:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.511436:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.552184:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972177 is committed 00002000:00010000:1.0:1789760745.552366:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19894272 -> 19898368 00010000:00010000:1.0:1789760745.552389:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552410:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552421:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552429:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.552434:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552444:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552469:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.552477:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b810a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.552645:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41321us (41634us total) trans 4294972177 rc 0/0 00000100:00100000:1.0:1789760745.552659:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4887 00000100:00100000:1.0:1789760745.552670:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726781005034 time 726826607131 state 0x2 ref 0 00000100:00100000:1.0:1789760745.555744:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.555834:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253248 00000100:00100000:3.0:1789760745.555847:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253248 00000100:00100000:3.0:1789760745.555921:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726781005034 time 726826607131 state 0x0 ref 1 00000100:00100000:3.0:1789760745.555940:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726826867547 time 726826607131 state 0x1 ref 1 00000100:00100000:3.0:1789760745.555975:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726826867547 time 726871277293 state 0x0 ref 1 00000100:00100000:3.0:1789760745.555990:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.556056:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19898368, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.556067:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.556087:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.556092:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.556099:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.556115:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.589213:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972178 is committed 00002000:00010000:3.0:1789760745.589411:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19898368 -> 19902464 00010000:00010000:3.0:1789760745.589433:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589449:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589459:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589467:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.589473:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589483:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589509:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.589518:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8111 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.589658:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33663us (33921us total) trans 4294972178 rc 0/0 00000100:00100000:3.0:1789760745.589671:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4888 00000100:00100000:3.0:1789760745.589684:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726826867547 time 726871277293 state 0x2 ref 0 00000100:00100000:2.0:1789760745.595680:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.596018:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253376 00000100:00100000:3.0:1789760745.596033:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253376 00000100:00100000:3.0:1789760745.596109:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726826867547 time 726871277293 state 0x0 ref 1 00000100:00100000:3.0:1789760745.596129:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726871537709 time 726871277293 state 0x1 ref 1 00000100:00100000:3.0:1789760745.596164:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726871537709 time 726911467357 state 0x0 ref 1 00000100:00100000:3.0:1789760745.596179:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.596378:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19902464, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.596390:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.596412:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.596417:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.596425:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.596442:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.628085:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972179 is committed 00002000:00010000:1.0:1789760745.628288:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19902464 -> 19906560 00010000:00010000:1.0:1789760745.628308:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628324:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628335:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628344:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.628349:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628359:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628386:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.628395:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8118 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.628525:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32340us (32851us total) trans 4294972179 rc 0/0 00000100:00100000:1.0:1789760745.628537:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4889 00000100:00100000:1.0:1789760745.628551:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726871537709 time 726911467357 state 0x2 ref 0 00000100:00100000:1.0:1789760745.631503:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.631606:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253504 00000100:00100000:3.0:1789760745.631619:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253504 00000100:00100000:3.0:1789760745.631720:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726871537709 time 726911467357 state 0x0 ref 1 00000100:00100000:3.0:1789760745.631740:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726911727773 time 726911467357 state 0x1 ref 1 00000100:00100000:3.0:1789760745.631781:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726911727773 time 726947083705 state 0x0 ref 1 00000100:00100000:3.0:1789760745.631805:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.631868:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19906560, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.631879:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.631898:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.631904:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.631911:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.631928:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.666962:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19906560 -> 19910656 00010000:00010000:3.0:1789760745.666992:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667009:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667020:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667028:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.667034:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667043:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667069:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667077:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b811f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.667222:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507253504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35421us (35725us total) trans 4294972180 rc 0/0 00000100:00100000:3.0:1789760745.667234:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4890 00000100:00100000:3.0:1789760745.667247:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726911727773 time 726947083705 state 0x2 ref 0 00000001:00080000:2.0:1789760745.669769:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972180 is committed 00000100:00100000:2.0:1789760745.671432:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.671526:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253632 00000100:00100000:3.0:1789760745.671541:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253632 00000100:00100000:3.0:1789760745.671619:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726911727773 time 726947083705 state 0x0 ref 1 00000100:00100000:3.0:1789760745.671638:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726947344121 time 726947083705 state 0x1 ref 1 00000100:00100000:3.0:1789760745.671677:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726947344121 time 726986979856 state 0x0 ref 1 00000100:00100000:3.0:1789760745.671734:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.671975:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19910656, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.671987:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.672007:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760745.672013:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.672021:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.672038:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.693988:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760745.694019:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108041500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538276096:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760745.694109:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760745.694192:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276096 00000100:00100000:1.0:1789760745.694322:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276096 00000100:00100000:1.0:1789760745.694612:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1789760745.694625:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538276096:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760745.694707:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760745.694782:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538276096:12345-0@lo:41:osp-pre-0-1.0 Request processed in 159us (672us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760745.694790:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1789760745.699837:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108041500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538276096:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760745.711125:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972181 is committed 00002000:00010000:3.0:1789760745.711369:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19910656 -> 19914752 00010000:00010000:3.0:1789760745.711389:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711405:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711415:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711423:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760745.711428:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711439:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711472:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.711480:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8126 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.711660:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39964us (40234us total) trans 4294972181 rc 0/0 00000100:00100000:3.0:1789760745.711671:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4891 00000100:00100000:3.0:1789760745.711682:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726947344121 time 726986979856 state 0x2 ref 0 00000100:00100000:1.0:1789760745.714833:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760745.715004:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253760 00000100:00100000:2.0:1789760745.715017:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253760 00000100:00100000:2.0:1789760745.715091:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726947344121 time 726986979856 state 0x0 ref 1 00000100:00100000:2.0:1789760745.715110:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726987240272 time 726986979856 state 0x1 ref 1 00000100:00100000:2.0:1789760745.715143:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726987240272 time 727030446264 state 0x0 ref 1 00000100:00100000:2.0:1789760745.715158:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760745.715231:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19914752, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:2.0:1789760745.715242:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.715262:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760745.715267:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.715275:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.715291:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.761554:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19914752 -> 19918848 00010000:00010000:1.0:1789760745.761585:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761601:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761611:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761625:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.761631:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761641:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761665:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.761674:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b812d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.761843:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507253760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46680us (47017us total) trans 4294972182 rc 0/0 00000100:00100000:1.0:1789760745.761856:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4892 00000100:00100000:1.0:1789760745.761868:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726987240272 time 727030446264 state 0x2 ref 0 00000001:00080000:0.0:1789760745.762672:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972182 is committed 00000100:00100000:1.0:1789760745.765243:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.765426:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507253888 00000100:00100000:3.0:1789760745.765439:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507253888 00000100:00100000:3.0:1789760745.765514:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 726987240272 time 727030446264 state 0x0 ref 1 00000100:00100000:3.0:1789760745.765534:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727030706680 time 727030446264 state 0x1 ref 1 00000100:00100000:3.0:1789760745.765567:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727030706680 time 727080869940 state 0x0 ref 1 00000100:00100000:3.0:1789760745.765582:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.765653:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19918848, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.765664:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.765684:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760745.765689:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.765873:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.765891:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.799573:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972183 is committed 00002000:00010000:1.0:1789760745.799684:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19918848 -> 19922944 00010000:00010000:1.0:1789760745.799723:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.799739:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.800087:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.800100:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.800107:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.800119:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.800146:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.800155:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8134 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.800406:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507253888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34818us (35172us total) trans 4294972183 rc 0/0 00000100:00100000:1.0:1789760745.800419:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4893 00000100:00100000:1.0:1789760745.800432:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727030706680 time 727080869940 state 0x2 ref 0 00000100:00100000:2.0:1789760745.805424:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.805540:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254016 00000100:00100000:3.0:1789760745.805555:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254016 00000100:00100000:3.0:1789760745.805629:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727030706680 time 727080869940 state 0x0 ref 1 00000100:00100000:3.0:1789760745.805650:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727081130356 time 727080869940 state 0x1 ref 1 00000100:00100000:3.0:1789760745.805738:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727081130356 time 727121039781 state 0x0 ref 1 00000100:00100000:3.0:1789760745.805753:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.805962:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19922944, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x9807 00010000:00010000:3.0:1789760745.805974:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.805995:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760745.806000:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.806009:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.806025:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.872436:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972184 is committed 00002000:00010000:2.0:1789760745.872685:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19922944 -> 19927040 00002000:00010000:2.0:1789760745.872727:0:8092:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:2.0:1789760745.873038:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873064:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873075:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873084:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760745.873090:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873102:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873128:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873137:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b813b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.873365:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67606us (67953us total) trans 4294972184 rc 0/0 00000100:00100000:2.0:1789760745.873378:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4894 00000100:00100000:2.0:1789760745.873390:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727081130356 time 727121039781 state 0x2 ref 0 00000100:00100000:2.0:1789760745.878045:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760745.878368:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254144 00000100:00100000:1.0:1789760745.878383:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254144 00000100:00100000:1.0:1789760745.878463:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727081130356 time 727121039781 state 0x0 ref 1 00000100:00100000:1.0:1789760745.878482:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727121300197 time 727121039781 state 0x1 ref 1 00000100:00100000:1.0:1789760745.878519:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727121300197 time 727193821677 state 0x0 ref 1 00000100:00100000:1.0:1789760745.878533:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760745.878603:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19927040, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa007 00010000:00010000:1.0:1789760745.878614:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.878635:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760745.878641:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.878649:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.878667:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.930596:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972185 is committed 00002000:00010000:0.0:1789760745.930780:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19927040 -> 19931136 00010000:00010000:0.0:1789760745.930964:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.930982:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.930992:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.931001:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760745.931008:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.931019:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.931046:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.931054:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8142 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.931247:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52708us (53279us total) trans 4294972185 rc 0/0 00000100:00100000:0.0:1789760745.931261:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4895 00000100:00100000:0.0:1789760745.931274:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727121300197 time 727193821677 state 0x2 ref 0 00000100:00100000:1.0:1789760745.934574:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.934681:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254272 00000100:00100000:3.0:1789760745.934791:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254272 00000100:00100000:3.0:1789760745.934930:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727121300197 time 727193821677 state 0x0 ref 1 00000100:00100000:3.0:1789760745.934950:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727194082093 time 727193821677 state 0x1 ref 1 00000100:00100000:3.0:1789760745.934978:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727194082093 time 727250280929 state 0x0 ref 1 00000100:00100000:3.0:1789760745.934992:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.935059:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19931136, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa007 00010000:00010000:3.0:1789760745.935071:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.935091:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760745.935096:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.935104:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.935122:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.986183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972186 is committed 00002000:00010000:1.0:1789760745.986904:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19931136 -> 19935232 00010000:00010000:1.0:1789760745.986927:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.986944:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.986955:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.986970:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760745.986976:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.986987:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.987014:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.987024:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8149 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.987155:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52158us (52589us total) trans 4294972186 rc 0/0 00000100:00100000:1.0:1789760745.987168:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4896 00000100:00100000:1.0:1789760745.987181:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727194082093 time 727250280929 state 0x2 ref 0 00000100:00100000:2.0:1789760745.991228:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760745.991334:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254400 00000100:00100000:3.0:1789760745.991347:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254400 00000100:00100000:3.0:1789760745.991425:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727194082093 time 727250280929 state 0x0 ref 1 00000100:00100000:3.0:1789760745.991447:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727250541345 time 727250280929 state 0x1 ref 1 00000100:00100000:3.0:1789760745.991476:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727250541345 time 727306779386 state 0x0 ref 1 00000100:00100000:3.0:1789760745.991492:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760745.991560:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19935232, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa007 00010000:00010000:3.0:1789760745.991571:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.991592:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760745.991598:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.991606:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.991623:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.035184:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972187 is committed 00002000:00010000:0.0:1789760746.035357:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19935232 -> 19939328 00002000:00010000:0.0:1789760746.035362:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:0.0:1789760746.035365:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:0.0:1789760746.035382:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035398:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035409:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035417:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.035422:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035432:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035457:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.035464:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8150 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.035623:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44127us (44402us total) trans 4294972187 rc 0/0 00000100:00100000:0.0:1789760746.035637:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4897 00000100:00100000:0.0:1789760746.035650:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727250541345 time 727306779386 state 0x2 ref 0 00000100:00100000:2.0:1789760746.039294:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.039391:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254528 00000100:00100000:3.0:1789760746.039405:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254528 00000100:00100000:3.0:1789760746.039478:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727250541345 time 727306779386 state 0x0 ref 1 00000100:00100000:3.0:1789760746.039495:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727307039802 time 727306779386 state 0x1 ref 1 00000100:00100000:3.0:1789760746.039523:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727307039802 time 727354826255 state 0x0 ref 1 00000100:00100000:3.0:1789760746.039536:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.039600:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19939328, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.039611:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.039629:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.039634:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.039642:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.039659:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.083165:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972188 is committed 00002000:00010000:0.0:1789760746.083349:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19939328 -> 19943424 00010000:00010000:0.0:1789760746.083371:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083387:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083398:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083406:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.083411:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083421:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083445:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.083455:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8157 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.083579:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44037us (44292us total) trans 4294972188 rc 0/0 00000100:00100000:0.0:1789760746.083591:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4898 00000100:00100000:0.0:1789760746.083604:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727307039802 time 727354826255 state 0x2 ref 0 00000100:00100000:1.0:1789760746.086739:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.086895:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254656 00000100:00100000:3.0:1789760746.086908:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254656 00000100:00100000:3.0:1789760746.086979:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727307039802 time 727354826255 state 0x0 ref 1 00000100:00100000:3.0:1789760746.086999:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727355086671 time 727354826255 state 0x1 ref 1 00000100:00100000:3.0:1789760746.087031:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727355086671 time 727402334261 state 0x0 ref 1 00000100:00100000:3.0:1789760746.087044:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.087106:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19943424, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.087117:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.087135:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.087142:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.087150:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.087167:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.127072:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972189 is committed 00002000:00010000:0.0:1789760746.127287:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19943424 -> 19947520 00010000:00010000:0.0:1789760746.127309:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127324:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127335:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127343:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.127349:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127359:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127384:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.127398:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b815e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.127561:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40511us (40829us total) trans 4294972189 rc 0/0 00000100:00100000:0.0:1789760746.127573:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4899 00000100:00100000:0.0:1789760746.127585:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727355086671 time 727402334261 state 0x2 ref 0 00000100:00100000:2.0:1789760746.132053:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.132158:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254784 00000100:00100000:0.0:1789760746.132170:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254784 00000100:00100000:0.0:1789760746.132311:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727355086671 time 727402334261 state 0x0 ref 1 00000100:00100000:0.0:1789760746.132331:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727402594677 time 727402334261 state 0x1 ref 1 00000100:00100000:0.0:1789760746.132408:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727402594677 time 727447710838 state 0x0 ref 1 00000100:00100000:0.0:1789760746.132422:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760746.132490:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19947520, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760746.132502:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.132521:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760746.132526:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.132533:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.132556:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.167066:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19947520 -> 19951616 00010000:00010000:0.0:1789760746.167092:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167109:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167119:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167128:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.167134:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167145:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167174:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.167182:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8165 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.167331:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972190 is committed 00000100:00100000:0.0:1789760746.167374:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507254784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34945us (35330us total) trans 4294972190 rc 0/0 00000100:00100000:0.0:1789760746.167387:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4900 00000100:00100000:0.0:1789760746.167398:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727402594677 time 727447710838 state 0x2 ref 0 00000100:00100000:1.0:1789760746.170517:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.170624:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507254912 00000100:00100000:0.0:1789760746.170638:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507254912 00000100:00100000:0.0:1789760746.170778:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727402594677 time 727447710838 state 0x0 ref 1 00000100:00100000:0.0:1789760746.170850:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727447971254 time 727447710838 state 0x1 ref 1 00000100:00100000:1.0:1789760746.171018:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727447971254 time 727486319549 state 0x0 ref 1 00000100:00100000:1.0:1789760746.171035:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760746.171100:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19951616, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:1.0:1789760746.171112:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.171131:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760746.171137:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.171144:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.171161:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.199093:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972191 is committed 00002000:00010000:2.0:1789760746.199338:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19951616 -> 19955712 00010000:00010000:2.0:1789760746.199360:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199376:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199387:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199396:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760746.199401:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199412:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199438:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.199446:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b816c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.199614:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507254912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28574us (29104us total) trans 4294972191 rc 0/0 00000100:00100000:2.0:1789760746.199627:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4901 00000100:00100000:2.0:1789760746.199645:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727447971254 time 727486319549 state 0x2 ref 0 00000100:00100000:1.0:1789760746.208098:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.208189:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255040 00000100:00100000:0.0:1789760746.208203:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255040 00000100:00100000:0.0:1789760746.208284:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727447971254 time 727486319549 state 0x0 ref 1 00000100:00100000:0.0:1789760746.208305:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727486579965 time 727486319549 state 0x1 ref 1 00000100:00100000:0.0:1789760746.208337:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727486579965 time 727523640552 state 0x0 ref 1 00000100:00100000:0.0:1789760746.208351:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760746.208415:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19955712, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760746.208425:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.208446:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760746.208452:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.208459:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.208476:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.242581:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19955712 -> 19959808 00010000:00010000:1.0:1789760746.242608:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242624:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242635:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242643:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760746.242649:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242660:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242686:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.242726:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8173 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.242863:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507255040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34506us (34770us total) trans 4294972192 rc 0/0 00000100:00100000:1.0:1789760746.242877:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4902 00000100:00100000:1.0:1789760746.242891:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727486579965 time 727523640552 state 0x2 ref 0 00000001:00080000:2.0:1789760746.243456:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972192 is committed 00000100:00100000:3.0:1789760746.250439:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.250612:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255168 00000100:00100000:0.0:1789760746.250625:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255168 00000100:00100000:0.0:1789760746.250742:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727486579965 time 727523640552 state 0x0 ref 1 00000100:00100000:0.0:1789760746.250771:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727523900968 time 727523640552 state 0x1 ref 1 00000100:00100000:2.0:1789760746.250846:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727523900968 time 727566144957 state 0x0 ref 1 00000100:00100000:2.0:1789760746.250864:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760746.250931:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19959808, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:2.0:1789760746.250942:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.250961:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760746.250967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.250981:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.250999:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.285614:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972193 is committed 00002000:00010000:3.0:1789760746.285821:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19959808 -> 19963904 00010000:00010000:3.0:1789760746.285843:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285859:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285869:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285876:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.285882:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285890:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285916:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.285922:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b817a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.286038:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35171us (35609us total) trans 4294972193 rc 0/0 00000100:00100000:3.0:1789760746.286049:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4903 00000100:00100000:3.0:1789760746.286060:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727523900968 time 727566144957 state 0x2 ref 0 00000100:00100000:3.0:1789760746.291013:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760746.291376:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255296 00000100:00100000:2.0:1789760746.291391:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255296 00000100:00100000:2.0:1789760746.291472:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727523900968 time 727566144957 state 0x0 ref 1 00000100:00100000:2.0:1789760746.291496:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727566405373 time 727566144957 state 0x1 ref 1 00000100:00100000:2.0:1789760746.291586:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727566405373 time 727606887639 state 0x0 ref 1 00000100:00100000:2.0:1789760746.291603:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760746.291675:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19963904, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:2.0:1789760746.291688:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.291766:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760746.291773:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.291782:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.292093:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.322441:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972194 is committed 00002000:00010000:3.0:1789760746.322746:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19963904 -> 19968000 00010000:00010000:3.0:1789760746.322776:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.322950:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.322962:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.322970:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.322977:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.322987:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.323017:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.323025:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8181 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.323217:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31609us (32212us total) trans 4294972194 rc 0/0 00000100:00100000:2.0:1789760746.323230:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4904 00000100:00100000:2.0:1789760746.323243:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727566405373 time 727606887639 state 0x2 ref 0 00000100:00100000:1.0:1789760746.330447:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.331794:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255424 00000100:00100000:0.0:1789760746.331813:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255424 00000100:00100000:2.0:1789760746.332051:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727566405373 time 727606887639 state 0x0 ref 1 00000100:00100000:2.0:1789760746.332071:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727607148055 time 727606887639 state 0x1 ref 1 00000100:00100000:2.0:1789760746.332253:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727607148055 time 727647555057 state 0x0 ref 1 00000100:00100000:2.0:1789760746.332271:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760746.332352:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19968000, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:2.0:1789760746.332363:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.332385:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760746.332390:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.332399:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.332415:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.369783:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972195 is committed 00002000:00010000:3.0:1789760746.369984:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19968000 -> 19972096 00010000:00010000:3.0:1789760746.370005:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370023:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370034:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370041:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.370047:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370056:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370079:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.370092:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8188 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.370240:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37968us (39806us total) trans 4294972195 rc 0/0 00000100:00100000:3.0:1789760746.370252:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4905 00000100:00100000:3.0:1789760746.370265:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727607148055 time 727647555057 state 0x2 ref 0 00000100:00100000:3.0:1789760746.372874:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760746.373022:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255552 00000100:00100000:2.0:1789760746.373034:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255552 00000100:00100000:2.0:1789760746.373104:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727607148055 time 727647555057 state 0x0 ref 1 00000100:00100000:2.0:1789760746.373122:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727647815473 time 727647555057 state 0x1 ref 1 00000100:00100000:2.0:1789760746.373158:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727647815473 time 727688461655 state 0x0 ref 1 00000100:00100000:2.0:1789760746.373171:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760746.373241:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19972096, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:2.0:1789760746.373251:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.373271:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760746.373275:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.373283:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.373299:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.395914:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760746.406622:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972196 is committed 00002000:00010000:3.0:1789760746.406796:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19972096 -> 19976192 00010000:00010000:3.0:1789760746.406817:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406832:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406841:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406849:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.406855:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406866:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406893:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.406902:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b818f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.407029:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33851us (34160us total) trans 4294972196 rc 0/0 00000100:00100000:3.0:1789760746.407042:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4906 00000100:00100000:3.0:1789760746.407054:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727647815473 time 727688461655 state 0x2 ref 0 00000100:00100000:3.0:1789760746.410510:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760746.410623:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255680 00000100:00100000:2.0:1789760746.410636:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255680 00000100:00100000:2.0:1789760746.410890:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727647815473 time 727688461655 state 0x0 ref 1 00000100:00100000:2.0:1789760746.410911:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727688722071 time 727688461655 state 0x1 ref 1 00000100:00100000:2.0:1789760746.410984:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727688722071 time 727726286952 state 0x0 ref 1 00000100:00100000:2.0:1789760746.411000:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760746.411069:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19976192, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:2.0:1789760746.411081:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.411100:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760746.411106:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.411114:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.411131:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.441935:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972197 is committed 00002000:00010000:2.0:1789760746.442114:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19976192 -> 19980288 00010000:00010000:2.0:1789760746.442135:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442151:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442161:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442169:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760746.442174:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442185:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442281:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.442291:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8196 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.442456:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31453us (31955us total) trans 4294972197 rc 0/0 00000100:00100000:2.0:1789760746.442469:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4907 00000100:00100000:2.0:1789760746.442482:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727688722071 time 727726286952 state 0x2 ref 0 00000100:00100000:2.0:1789760746.445398:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.445458:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255808 00000100:00100000:3.0:1789760746.445471:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255808 00000100:00100000:3.0:1789760746.445545:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727688722071 time 727726286952 state 0x0 ref 1 00000100:00100000:3.0:1789760746.445565:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727726547368 time 727726286952 state 0x1 ref 1 00000100:00100000:3.0:1789760746.445648:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727726547368 time 727760949903 state 0x0 ref 1 00000100:00100000:3.0:1789760746.445662:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.445748:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19980288, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.445768:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.445887:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.445894:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.445903:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.445922:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.487033:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972198 is committed 00002000:00010000:3.0:1789760746.487163:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19980288 -> 19984384 00010000:00010000:3.0:1789760746.487183:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487209:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487219:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487227:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.487233:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487244:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487270:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.487279:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b819d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.487418:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507255808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41751us (42026us total) trans 4294972198 rc 0/0 00000100:00100000:3.0:1789760746.487431:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4908 00000100:00100000:3.0:1789760746.487442:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727726547368 time 727760949903 state 0x2 ref 0 00000100:00100000:1.0:1789760746.490443:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.490787:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507255936 00000100:00100000:3.0:1789760746.490852:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507255936 00000100:00100000:3.0:1789760746.490924:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727726547368 time 727760949903 state 0x0 ref 1 00000100:00100000:3.0:1789760746.490944:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727761210319 time 727760949903 state 0x1 ref 1 00000100:00100000:3.0:1789760746.490980:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727761210319 time 727806282680 state 0x0 ref 1 00000100:00100000:3.0:1789760746.490994:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.491061:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19984384, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.491073:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.491092:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.491097:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.491105:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.491128:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.519029:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972199 is committed 00002000:00010000:3.0:1789760746.519140:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19984384 -> 19988480 00010000:00010000:3.0:1789760746.519168:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519180:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519187:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519192:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.519196:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519203:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519222:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.519227:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.519343:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507255936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28344us (28909us total) trans 4294972199 rc 0/0 00000100:00100000:3.0:1789760746.519356:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4909 00000100:00100000:3.0:1789760746.519369:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727761210319 time 727806282680 state 0x2 ref 0 00000100:00100000:2.0:1789760746.525766:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.525821:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256064 00000100:00100000:0.0:1789760746.525836:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256064 00000100:00100000:0.0:1789760746.525921:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727761210319 time 727806282680 state 0x0 ref 1 00000100:00100000:0.0:1789760746.525940:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727806543096 time 727806282680 state 0x1 ref 1 00000100:00100000:0.0:1789760746.525975:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727806543096 time 727841278601 state 0x0 ref 1 00000100:00100000:0.0:1789760746.525989:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760746.526053:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19988480, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760746.526064:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.526083:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760746.526089:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.526096:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.526113:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.573595:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19988480 -> 19992576 00010000:00010000:0.0:1789760746.573622:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573638:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573647:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573655:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.573660:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573671:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573721:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.573730:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.574090:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972200 is committed 00000100:00100000:0.0:1789760746.574154:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48159us (48453us total) trans 4294972200 rc 0/0 00000100:00100000:0.0:1789760746.574167:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4910 00000100:00100000:0.0:1789760746.574179:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727806543096 time 727841278601 state 0x2 ref 0 00000100:00100000:2.0:1789760746.579330:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.579873:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256192 00000100:00100000:3.0:1789760746.579887:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256192 00000100:00100000:3.0:1789760746.579962:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727806543096 time 727841278601 state 0x0 ref 1 00000100:00100000:3.0:1789760746.579982:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727841539017 time 727841278601 state 0x1 ref 1 00000100:00100000:3.0:1789760746.580055:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727841539017 time 727895357084 state 0x0 ref 1 00000100:00100000:3.0:1789760746.580070:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.580149:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19992576, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.580160:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.580179:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.580184:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.580192:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.580209:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.633158:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19992576 -> 19996672 00010000:00010000:3.0:1789760746.633188:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633205:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633283:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633292:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760746.633297:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633308:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633334:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.633343:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.633557:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507256192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53482us (54244us total) trans 4294972201 rc 0/0 00000100:00100000:3.0:1789760746.633576:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4911 00000100:00100000:3.0:1789760746.633588:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727841539017 time 727895357084 state 0x2 ref 0 00000001:00080000:0.0:1789760746.633998:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972201 is committed 00000100:00100000:2.0:1789760746.637388:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.637478:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256320 00000100:00100000:3.0:1789760746.637492:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256320 00000100:00100000:3.0:1789760746.637567:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727841539017 time 727895357084 state 0x0 ref 1 00000100:00100000:3.0:1789760746.637585:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727895617500 time 727895357084 state 0x1 ref 1 00000100:00100000:3.0:1789760746.637622:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727895617500 time 727952925436 state 0x0 ref 1 00000100:00100000:3.0:1789760746.637636:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.637746:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19996672, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.637766:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.637785:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760746.637790:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.637798:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.637815:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.692560:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972202 is committed 00002000:00010000:2.0:1789760746.692603:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19996672 -> 20000768 00010000:00010000:2.0:1789760746.692625:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692641:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692652:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692660:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760746.692666:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692676:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692873:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.692884:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b81b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.693087:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55446us (55708us total) trans 4294972202 rc 0/0 00000100:00100000:2.0:1789760746.693100:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4912 00000100:00100000:2.0:1789760746.693113:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727895617500 time 727952925436 state 0x2 ref 0 00000100:00100000:2.0:1789760746.703889:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.703949:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256448 00000100:00100000:0.0:1789760746.703974:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256448 00000100:00100000:0.0:1789760746.704053:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727895617500 time 727952925436 state 0x0 ref 1 00000100:00100000:0.0:1789760746.704083:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727953185852 time 727952925436 state 0x1 ref 1 00000100:00100000:0.0:1789760746.704118:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727953185852 time 728019421316 state 0x0 ref 1 00000100:00100000:0.0:1789760746.704132:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760746.704196:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20000768, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760746.704207:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.704233:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760746.704239:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.704246:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.704262:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.798366:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972203 is committed 00002000:00010000:0.0:1789760746.798660:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20000768 -> 20004864 00010000:00010000:0.0:1789760746.798680:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798719:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798730:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798738:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.798744:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798764:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798966:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.798978:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.799114:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 94975us (95234us total) trans 4294972203 rc 0/0 00000100:00100000:0.0:1789760746.799127:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4913 00000100:00100000:0.0:1789760746.799140:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727953185852 time 728019421316 state 0x2 ref 0 00000100:00100000:1.0:1789760746.803121:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.803288:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256576 00000100:00100000:3.0:1789760746.803301:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256576 00000100:00100000:3.0:1789760746.803375:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 727953185852 time 728019421316 state 0x0 ref 1 00000100:00100000:3.0:1789760746.803394:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728019681732 time 728019421316 state 0x1 ref 1 00000100:00100000:3.0:1789760746.803466:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728019681732 time 728118768592 state 0x0 ref 1 00000100:00100000:3.0:1789760746.803553:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.803629:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20004864, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.803641:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.803661:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760746.803667:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.803676:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.803723:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.867104:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972204 is committed 00002000:00010000:0.0:1789760746.867297:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20004864 -> 20008960 00010000:00010000:0.0:1789760746.867316:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867331:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867341:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867349:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760746.867354:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867365:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867395:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.867404:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b81c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.867520:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64036us (64406us total) trans 4294972204 rc 0/0 00000100:00100000:0.0:1789760746.867531:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4914 00000100:00100000:0.0:1789760746.867542:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728019681732 time 728118768592 state 0x2 ref 0 00000100:00100000:2.0:1789760746.878077:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760746.878255:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256704 00000100:00100000:3.0:1789760746.878267:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256704 00000100:00100000:3.0:1789760746.878351:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728019681732 time 728118768592 state 0x0 ref 1 00000100:00100000:3.0:1789760746.878370:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728119029008 time 728118768592 state 0x1 ref 1 00000100:00100000:3.0:1789760746.878403:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728119029008 time 728193705638 state 0x0 ref 1 00000100:00100000:3.0:1789760746.878421:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760746.878494:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20008960, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:3.0:1789760746.878505:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.878525:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760746.878530:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.878544:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.878562:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.931249:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972205 is committed 00002000:00010000:2.0:1789760746.931522:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20008960 -> 20013056 00010000:00010000:2.0:1789760746.931543:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931559:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931570:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931579:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760746.931586:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931596:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931622:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.931630:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b81ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.932028:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507256704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53601us (53960us total) trans 4294972205 rc 0/0 00000100:00100000:2.0:1789760746.932042:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4915 00000100:00100000:2.0:1789760746.932056:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728119029008 time 728193705638 state 0x2 ref 0 00000100:00100000:3.0:1789760746.937832:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760746.937930:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256832 00000100:00100000:0.0:1789760746.938008:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256832 00000100:00100000:0.0:1789760746.938082:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728119029008 time 728193705638 state 0x0 ref 1 00000100:00100000:0.0:1789760746.938100:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728193966054 time 728193705638 state 0x1 ref 1 00000100:00100000:0.0:1789760746.938137:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728193966054 time 728253439416 state 0x0 ref 1 00000100:00100000:0.0:1789760746.938151:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760746.938520:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20013056, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760746.938532:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.938553:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760746.938560:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.938568:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.938586:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.995759:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972206 is committed 00002000:00010000:1.0:1789760746.995990:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20013056 -> 20017152 00010000:00010000:1.0:1789760746.996012:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996028:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996038:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996046:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760746.996052:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996061:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996102:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996111:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b81d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.996352:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507256832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58195us (58526us total) trans 4294972206 rc 0/0 00000100:00100000:1.0:1789760746.996364:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4916 00000100:00100000:1.0:1789760746.996376:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728193966054 time 728253439416 state 0x2 ref 0 00000100:00100000:3.0:1789760746.999666:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.000040:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507256960 00000100:00100000:0.0:1789760747.000053:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507256960 00000100:00100000:0.0:1789760747.000215:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728193966054 time 728253439416 state 0x0 ref 1 00000100:00100000:0.0:1789760747.000233:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728253699832 time 728253439416 state 0x1 ref 1 00000100:00100000:0.0:1789760747.000271:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728253699832 time 728315574724 state 0x0 ref 1 00000100:00100000:0.0:1789760747.000285:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507256960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.000348:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20017152, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa007 00010000:00010000:0.0:1789760747.000358:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.000378:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.000383:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.000391:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.000407:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.049678:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20017152 -> 20021248 00002000:00010000:0.0:1789760747.049737:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:0.0:1789760747.049741:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:0.0:1789760747.049786:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049802:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049812:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049819:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.049825:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049834:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049860:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.049869:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.050067:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507256960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49775us (50403us total) trans 4294972207 rc 0/0 00000100:00100000:2.0:1789760747.050081:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4917 00000100:00100000:2.0:1789760747.050096:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728253699832 time 728315574724 state 0x2 ref 0 00000001:00080000:3.0:1789760747.050957:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972207 is committed 00000100:00100000:2.0:1789760747.054032:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.054145:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257088 00000100:00100000:0.0:1789760747.054158:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257088 00000100:00100000:0.0:1789760747.054253:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728253699832 time 728315574724 state 0x0 ref 1 00000100:00100000:0.0:1789760747.054273:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728315835140 time 728315574724 state 0x1 ref 1 00000100:00100000:0.0:1789760747.054310:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728315835140 time 728369612577 state 0x0 ref 1 00000100:00100000:0.0:1789760747.054324:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507257088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.054398:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20021248, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.054410:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.054432:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.054437:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.054445:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.054461:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760747.055071:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760747.103558:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972208 is committed 00002000:00010000:0.0:1789760747.103729:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20021248 -> 20025344 00010000:00010000:0.0:1789760747.103769:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103803:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103815:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103824:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.103829:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103839:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103866:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.103874:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b81e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.104106:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507257088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49777us (50082us total) trans 4294972208 rc 0/0 00000100:00100000:0.0:1789760747.104270:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4918 00000100:00100000:0.0:1789760747.104283:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728315835140 time 728369612577 state 0x2 ref 0 00000100:00100000:0.0:1789760747.114481:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760747.116326:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257216 00000100:00100000:1.0:1789760747.116351:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257216 00000100:00100000:1.0:1789760747.116439:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728315835140 time 728369612577 state 0x0 ref 1 00000100:00100000:1.0:1789760747.116458:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728369872993 time 728369612577 state 0x1 ref 1 00000100:00100000:1.0:1789760747.116502:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728369872993 time 728431798996 state 0x0 ref 1 00000100:00100000:1.0:1789760747.116520:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760747.116584:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20025344, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:1.0:1789760747.116596:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.116616:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760747.116621:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.116628:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.116645:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.172650:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972209 is committed 00002000:00010000:2.0:1789760747.172915:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20025344 -> 20029440 00010000:00010000:2.0:1789760747.172938:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.172959:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.172970:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.172978:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760747.172984:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.172994:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173021:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173030:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b81ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.173212:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56687us (58739us total) trans 4294972209 rc 0/0 00000100:00100000:2.0:1789760747.173226:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4919 00000100:00100000:2.0:1789760747.173250:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728369872993 time 728431798996 state 0x2 ref 0 00000100:00100000:2.0:1789760747.181154:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760747.183500:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257344 00000100:00100000:3.0:1789760747.183521:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257344 00000100:00100000:3.0:1789760747.183608:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728369872993 time 728431798996 state 0x0 ref 1 00000100:00100000:3.0:1789760747.183628:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728432059412 time 728431798996 state 0x1 ref 1 00000100:00100000:3.0:1789760747.183786:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728432059412 time 728499087410 state 0x0 ref 1 00000100:00100000:3.0:1789760747.183808:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507257344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760747.183881:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20029440, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:3.0:1789760747.183892:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.183914:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760747.183919:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.183927:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.183946:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.238158:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972210 is committed 00002000:00010000:2.0:1789760747.238359:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20029440 -> 20033536 00010000:00010000:2.0:1789760747.238379:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238395:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238405:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238413:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760747.238419:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238430:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238456:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.238464:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b81f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.238592:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507257344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54780us (57445us total) trans 4294972210 rc 0/0 00000100:00100000:2.0:1789760747.238605:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4920 00000100:00100000:2.0:1789760747.238617:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728432059412 time 728499087410 state 0x2 ref 0 00000100:00100000:1.0:1789760747.241663:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.241983:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257472 00000100:00100000:0.0:1789760747.241997:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257472 00000100:00100000:0.0:1789760747.242073:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728432059412 time 728499087410 state 0x0 ref 1 00000100:00100000:0.0:1789760747.242094:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728499347826 time 728499087410 state 0x1 ref 1 00000100:00100000:0.0:1789760747.242162:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728499347826 time 728557463755 state 0x0 ref 1 00000100:00100000:0.0:1789760747.242178:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.242326:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20033536, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.242338:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.242359:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.242365:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.242374:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.242393:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.278046:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972211 is committed 00002000:00010000:0.0:1789760747.278135:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20033536 -> 20037632 00010000:00010000:0.0:1789760747.278155:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278178:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278199:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278207:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.278214:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278225:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278251:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.278258:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b81f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.278387:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36205us (36731us total) trans 4294972211 rc 0/0 00000100:00100000:0.0:1789760747.278400:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4921 00000100:00100000:0.0:1789760747.278412:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728499347826 time 728557463755 state 0x2 ref 0 00000100:00100000:0.0:1789760747.282542:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760747.282674:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257600 00000100:00100000:2.0:1789760747.282688:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257600 00000100:00100000:2.0:1789760747.283024:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728499347826 time 728557463755 state 0x0 ref 1 00000100:00100000:2.0:1789760747.283044:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728557724171 time 728557463755 state 0x1 ref 1 00000100:00100000:2.0:1789760747.283075:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728557724171 time 728598378077 state 0x0 ref 1 00000100:00100000:2.0:1789760747.283091:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507257600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760747.283159:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20037632, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:2.0:1789760747.283171:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.283204:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760747.283210:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.283218:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.283235:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.316494:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972212 is committed 00002000:00010000:2.0:1789760747.316643:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20037632 -> 20041728 00010000:00010000:2.0:1789760747.316664:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316680:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316721:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316731:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760747.316737:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316758:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316855:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.316865:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b81ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.317073:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507257600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33977us (34539us total) trans 4294972212 rc 0/0 00000100:00100000:2.0:1789760747.317086:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4922 00000100:00100000:2.0:1789760747.317099:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728557724171 time 728598378077 state 0x2 ref 0 00000100:00100000:1.0:1789760747.320574:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760747.320685:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257728 00000100:00100000:2.0:1789760747.320843:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257728 00000100:00100000:0.0:1789760747.321010:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728557724171 time 728598378077 state 0x0 ref 1 00000100:00100000:0.0:1789760747.321030:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728598638493 time 728598378077 state 0x1 ref 1 00000100:00100000:0.0:1789760747.321106:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728598638493 time 728636406918 state 0x0 ref 1 00000100:00100000:0.0:1789760747.321122:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.321263:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20041728, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.321277:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.321298:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.321303:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.321311:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.321330:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.355568:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972213 is committed 00002000:00010000:0.0:1789760747.355723:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20041728 -> 20045824 00010000:00010000:0.0:1789760747.355755:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355772:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355851:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355862:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.355869:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355880:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355905:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.355915:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8206 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.356046:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507257728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34920us (35478us total) trans 4294972213 rc 0/0 00000100:00100000:0.0:1789760747.356059:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4923 00000100:00100000:0.0:1789760747.356073:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728598638493 time 728636406918 state 0x2 ref 0 00000100:00100000:1.0:1789760747.360053:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.360151:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257856 00000100:00100000:0.0:1789760747.360164:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257856 00000100:00100000:0.0:1789760747.360304:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728598638493 time 728636406918 state 0x0 ref 1 00000100:00100000:0.0:1789760747.360322:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728636667334 time 728636406918 state 0x1 ref 1 00000100:00100000:0.0:1789760747.360397:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728636667334 time 728675699525 state 0x0 ref 1 00000100:00100000:0.0:1789760747.360414:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507257856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.360481:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20045824, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.360492:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.360513:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.360520:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.360528:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.360546:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.397105:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20045824 -> 20049920 00010000:00010000:0.0:1789760747.397132:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397148:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397158:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397167:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.397173:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397186:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397223:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.397232:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b820d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.397362:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507257856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36942us (37316us total) trans 4294972214 rc 0/0 00000100:00100000:0.0:1789760747.397374:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4924 00000100:00100000:0.0:1789760747.397386:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728636667334 time 728675699525 state 0x2 ref 0 00000100:00100000:0.0:1789760747.400715:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760747.400931:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507257984 00000100:00100000:3.0:1789760747.400945:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507257984 00000100:00100000:3.0:1789760747.401022:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728636667334 time 728675699525 state 0x0 ref 1 00000100:00100000:3.0:1789760747.401041:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728675959941 time 728675699525 state 0x1 ref 1 00000100:00100000:3.0:1789760747.401076:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728675959941 time 728716378688 state 0x0 ref 1 00000100:00100000:3.0:1789760747.401092:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507257984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760747.401160:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20049920, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:3.0:1789760747.401172:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.401198:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760747.401204:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.401212:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.401230:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.401869:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972214 is committed 00010000:00010000:1.0:1789760747.423281:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760747.435340:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20049920 -> 20054016 00010000:00010000:3.0:1789760747.435369:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435385:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435395:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435403:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760747.435409:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435420:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435446:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.435455:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8214 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.435621:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507257984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34525us (34937us total) trans 4294972215 rc 0/0 00000100:00100000:3.0:1789760747.435633:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4925 00000100:00100000:3.0:1789760747.435644:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728675959941 time 728716378688 state 0x2 ref 0 00000001:00080000:1.0:1789760747.435816:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972215 is committed 00000100:00100000:2.0:1789760747.439988:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.440118:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258112 00000100:00100000:0.0:1789760747.440131:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258112 00000100:00100000:0.0:1789760747.440265:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728675959941 time 728716378688 state 0x0 ref 1 00000100:00100000:0.0:1789760747.440285:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728716639104 time 728716378688 state 0x1 ref 1 00000100:00100000:0.0:1789760747.440372:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728716639104 time 728755673420 state 0x0 ref 1 00000100:00100000:0.0:1789760747.440388:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.440455:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20054016, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.440467:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.440488:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.440494:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.440502:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.440519:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.474820:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972216 is committed 00002000:00010000:3.0:1789760747.475031:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20054016 -> 20058112 00010000:00010000:3.0:1789760747.475051:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475068:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475078:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475087:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760747.475093:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475102:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475134:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475142:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b821b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.475395:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35003us (35414us total) trans 4294972216 rc 0/0 00000100:00100000:3.0:1789760747.475408:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4926 00000100:00100000:3.0:1789760747.475421:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728716639104 time 728755673420 state 0x2 ref 0 00000100:00100000:0.0:1789760747.478990:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760747.479102:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258240 00000100:00100000:2.0:1789760747.479115:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258240 00000100:00100000:2.0:1789760747.479252:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728716639104 time 728755673420 state 0x0 ref 1 00000100:00100000:2.0:1789760747.479274:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728755933836 time 728755673420 state 0x1 ref 1 00000100:00100000:2.0:1789760747.479343:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728755933836 time 728794645261 state 0x0 ref 1 00000100:00100000:2.0:1789760747.479358:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760747.479431:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20058112, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:2.0:1789760747.479443:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.479465:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760747.479470:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.479478:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.479495:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.513824:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972217 is committed 00002000:00010000:0.0:1789760747.514021:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20058112 -> 20062208 00010000:00010000:0.0:1789760747.514045:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514062:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514073:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514081:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.514087:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514097:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514124:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.514133:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8222 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.514333:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34970us (35352us total) trans 4294972217 rc 0/0 00000100:00100000:0.0:1789760747.514346:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4927 00000100:00100000:0.0:1789760747.514359:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728755933836 time 728794645261 state 0x2 ref 0 00000100:00100000:3.0:1789760747.517399:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760747.517902:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258368 00000100:00100000:2.0:1789760747.517917:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258368 00000100:00100000:2.0:1789760747.517994:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728755933836 time 728794645261 state 0x0 ref 1 00000100:00100000:2.0:1789760747.518015:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728794905677 time 728794645261 state 0x1 ref 1 00000100:00100000:2.0:1789760747.518055:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728794905677 time 728833356942 state 0x0 ref 1 00000100:00100000:2.0:1789760747.518073:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507258368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760747.518143:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20062208, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:2.0:1789760747.518155:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.518176:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760747.518182:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.518198:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.518216:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.576575:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972218 is committed 00002000:00010000:3.0:1789760747.577003:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20062208 -> 20066304 00010000:00010000:3.0:1789760747.577152:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577168:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577179:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577186:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760747.577191:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577203:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577229:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.577238:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8229 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.577560:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507258368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59482us (60168us total) trans 4294972218 rc 0/0 00000100:00100000:3.0:1789760747.577573:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4928 00000100:00100000:3.0:1789760747.577586:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728794905677 time 728833356942 state 0x2 ref 0 00000100:00100000:2.0:1789760747.589838:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760747.591152:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258496 00000100:00100000:3.0:1789760747.591171:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258496 00000100:00100000:3.0:1789760747.591255:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728794905677 time 728833356942 state 0x0 ref 1 00000100:00100000:3.0:1789760747.591276:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728833617358 time 728833356942 state 0x1 ref 1 00000100:00100000:3.0:1789760747.591316:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728833617358 time 728906619305 state 0x0 ref 1 00000100:00100000:3.0:1789760747.591333:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760747.591405:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20066304, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:3.0:1789760747.591417:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.591441:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760747.591447:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.591455:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.591474:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.650555:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972219 is committed 00002000:00010000:3.0:1789760747.650678:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20066304 -> 20070400 00010000:00010000:3.0:1789760747.650727:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650753:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650764:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650772:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760747.650839:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650852:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650877:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.650886:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8230 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.651008:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59671us (61179us total) trans 4294972219 rc 0/0 00000100:00100000:3.0:1789760747.651020:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4929 00000100:00100000:3.0:1789760747.651032:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728833617358 time 728906619305 state 0x2 ref 0 00000100:00100000:1.0:1789760747.654769:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.654948:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258624 00000100:00100000:0.0:1789760747.654963:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258624 00000100:00100000:0.0:1789760747.655040:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728833617358 time 728906619305 state 0x0 ref 1 00000100:00100000:0.0:1789760747.655061:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728906879721 time 728906619305 state 0x1 ref 1 00000100:00100000:0.0:1789760747.655099:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728906879721 time 728970401638 state 0x0 ref 1 00000100:00100000:0.0:1789760747.655113:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507258624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.655175:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20070400, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.655186:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.655206:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760747.655212:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.655231:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.655249:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.696753:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972220 is committed 00002000:00010000:0.0:1789760747.696971:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20070400 -> 20074496 00010000:00010000:0.0:1789760747.696992:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697008:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697017:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697024:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.697030:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697039:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697067:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.697082:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8237 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.697397:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507258624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42278us (42645us total) trans 4294972220 rc 0/0 00000100:00100000:0.0:1789760747.697411:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4930 00000100:00100000:0.0:1789760747.697425:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728906879721 time 728970401638 state 0x2 ref 0 00000100:00100000:2.0:1789760747.708979:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760747.709080:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258752 00000100:00100000:3.0:1789760747.709092:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258752 00000100:00100000:3.0:1789760747.709178:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728906879721 time 728970401638 state 0x0 ref 1 00000100:00100000:3.0:1789760747.709198:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728970662054 time 728970401638 state 0x1 ref 1 00000100:00100000:3.0:1789760747.709238:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728970662054 time 729024540219 state 0x0 ref 1 00000100:00100000:3.0:1789760747.709252:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507258752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760747.709332:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20074496, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:3.0:1789760747.709344:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.709363:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760747.709369:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.709378:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.709397:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.771586:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20074496 -> 20078592 00010000:00010000:0.0:1789760747.771625:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771639:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771646:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771652:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.771656:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771662:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771686:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.771716:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b823e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.771926:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507258752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62668us (62955us total) trans 4294972221 rc 0/0 00000100:00100000:0.0:1789760747.771939:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4931 00000100:00100000:0.0:1789760747.771951:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728970662054 time 729024540219 state 0x2 ref 0 00000001:00080000:2.0:1789760747.772325:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972221 is committed 00000100:00100000:2.0:1789760747.785168:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760747.785605:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507258880 00000100:00100000:3.0:1789760747.785620:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507258880 00000100:00100000:3.0:1789760747.785920:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 728970662054 time 729024540219 state 0x0 ref 1 00000100:00100000:3.0:1789760747.785940:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729024800635 time 729024540219 state 0x1 ref 1 00000100:00100000:2.0:1789760747.786263:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729024800635 time 729101563085 state 0x0 ref 1 00000100:00100000:2.0:1789760747.786283:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507258880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760747.786354:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20078592, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:2.0:1789760747.786365:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.786389:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760747.786394:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.786403:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.786420:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.842164:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972222 is committed 00002000:00010000:0.0:1789760747.842344:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20078592 -> 20082688 00010000:00010000:0.0:1789760747.842365:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842379:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842389:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842396:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.842402:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842418:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842443:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.842451:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8245 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.842577:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507258880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56293us (57416us total) trans 4294972222 rc 0/0 00000100:00100000:0.0:1789760747.842589:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4932 00000100:00100000:0.0:1789760747.842601:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729024800635 time 729101563085 state 0x2 ref 0 00000100:00100000:1.0:1789760747.850115:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.850236:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259008 00000100:00100000:0.0:1789760747.850249:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259008 00000100:00100000:0.0:1789760747.850325:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729024800635 time 729101563085 state 0x0 ref 1 00000100:00100000:0.0:1789760747.850345:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729101823501 time 729101563085 state 0x1 ref 1 00000100:00100000:0.0:1789760747.850430:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729101823501 time 729165732646 state 0x0 ref 1 00000100:00100000:0.0:1789760747.850444:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.850513:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20082688, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.850525:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.850546:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760747.850551:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.850560:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.850577:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.894785:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20082688 -> 20086784 00010000:00010000:0.0:1789760747.894812:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894836:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894848:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894857:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.894863:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894873:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894898:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.894907:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b824c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.895018:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507259008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44568us (44976us total) trans 4294972223 rc 0/0 00000100:00100000:0.0:1789760747.895031:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4933 00000100:00100000:0.0:1789760747.895043:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729101823501 time 729165732646 state 0x2 ref 0 00000001:00080000:1.0:1789760747.896497:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972223 is committed 00000100:00100000:3.0:1789760747.901374:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.901485:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259136 00000100:00100000:0.0:1789760747.901500:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259136 00000100:00100000:0.0:1789760747.901578:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729101823501 time 729165732646 state 0x0 ref 1 00000100:00100000:0.0:1789760747.901598:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729165993062 time 729165732646 state 0x1 ref 1 00000100:00100000:0.0:1789760747.901679:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729165993062 time 729216981298 state 0x0 ref 1 00000100:00100000:0.0:1789760747.901719:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.901802:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20086784, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.901814:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.901842:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760747.901848:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.901856:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.901880:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.949460:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972224 is committed 00002000:00010000:0.0:1789760747.949597:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20086784 -> 20090880 00010000:00010000:0.0:1789760747.949618:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949633:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949644:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949652:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760747.949658:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949668:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949715:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.949724:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8253 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.950032:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48333us (48667us total) trans 4294972224 rc 0/0 00000100:00100000:0.0:1789760747.950046:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4934 00000100:00100000:0.0:1789760747.950067:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729165993062 time 729216981298 state 0x2 ref 0 00000100:00100000:2.0:1789760747.954385:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760747.954497:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259264 00000100:00100000:0.0:1789760747.954512:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259264 00000100:00100000:0.0:1789760747.954588:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729165993062 time 729216981298 state 0x0 ref 1 00000100:00100000:0.0:1789760747.954607:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729217241714 time 729216981298 state 0x1 ref 1 00000100:00100000:0.0:1789760747.954680:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729217241714 time 729269982790 state 0x0 ref 1 00000100:00100000:0.0:1789760747.954750:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507259264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760747.954821:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20090880, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760747.954839:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.954859:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760747.954865:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.954872:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.954890:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.002504:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20090880 -> 20094976 00010000:00010000:0.0:1789760748.002533:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002549:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002560:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002569:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.002576:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002587:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002612:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002620:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b825a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.002836:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507259264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48134us (48460us total) trans 4294972225 rc 0/0 00000100:00100000:0.0:1789760748.002853:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4935 00000100:00100000:0.0:1789760748.002865:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729217241714 time 729269982790 state 0x2 ref 0 00000001:00080000:2.0:1789760748.004965:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972225 is committed 00000100:00100000:1.0:1789760748.006357:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.006526:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259392 00000100:00100000:0.0:1789760748.006539:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259392 00000100:00100000:0.0:1789760748.006614:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729217241714 time 729269982790 state 0x0 ref 1 00000100:00100000:0.0:1789760748.006635:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729270243206 time 729269982790 state 0x1 ref 1 00000100:00100000:0.0:1789760748.006678:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729270243206 time 729321980981 state 0x0 ref 1 00000100:00100000:0.0:1789760748.006729:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.006819:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20094976, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa007 00010000:00010000:0.0:1789760748.006830:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.006850:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760748.006855:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.006863:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.006879:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.033651:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972226 is committed 00002000:00010000:0.0:1789760748.033913:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20094976 -> 20099072 00002000:00010000:0.0:1789760748.033919:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:0.0:1789760748.033923:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:0.0:1789760748.033942:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.033958:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.033968:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.033975:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.033981:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.033991:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034017:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034024:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8261 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.034142:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27444us (27788us total) trans 4294972226 rc 0/0 00000100:00100000:0.0:1789760748.034155:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4936 00000100:00100000:0.0:1789760748.034167:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729270243206 time 729321980981 state 0x2 ref 0 00000100:00100000:1.0:1789760748.039553:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.039948:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259520 00000100:00100000:0.0:1789760748.039962:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259520 00000100:00100000:0.0:1789760748.040038:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729270243206 time 729321980981 state 0x0 ref 1 00000100:00100000:0.0:1789760748.040057:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729322241397 time 729321980981 state 0x1 ref 1 00000100:00100000:0.0:1789760748.040092:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729322241397 time 729355394667 state 0x0 ref 1 00000100:00100000:0.0:1789760748.040106:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.040170:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20099072, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.040181:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.040200:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760748.040206:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.040268:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.040287:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.066601:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20099072 -> 20103168 00010000:00010000:2.0:1789760748.066626:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066643:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066653:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066660:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.066667:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066683:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066726:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.066736:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8268 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.066874:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507259520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26763us (27327us total) trans 4294972227 rc 0/0 00000100:00100000:2.0:1789760748.066886:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4937 00000100:00100000:2.0:1789760748.066898:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729322241397 time 729355394667 state 0x2 ref 0 00000001:00080000:1.0:1789760748.067050:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972227 is committed 00000100:00100000:0.0:1789760748.072651:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.073020:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259648 00000100:00100000:2.0:1789760748.073032:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259648 00000100:00100000:2.0:1789760748.073113:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729322241397 time 729355394667 state 0x0 ref 1 00000100:00100000:2.0:1789760748.073134:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729355655083 time 729355394667 state 0x1 ref 1 00000100:00100000:2.0:1789760748.073168:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729355655083 time 729388471062 state 0x0 ref 1 00000100:00100000:2.0:1789760748.073183:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.073249:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20103168, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.073259:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.073280:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.073286:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.073293:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.073311:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.115959:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972228 is committed 00002000:00010000:0.0:1789760748.115992:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20103168 -> 20107264 00010000:00010000:0.0:1789760748.116015:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116036:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116046:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116053:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.116058:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116068:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116092:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.116101:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b826f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.116359:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43171us (43716us total) trans 4294972228 rc 0/0 00000100:00100000:0.0:1789760748.116374:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4938 00000100:00100000:0.0:1789760748.116386:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729355655083 time 729388471062 state 0x2 ref 0 00000100:00100000:3.0:1789760748.126365:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.126671:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259776 00000100:00100000:0.0:1789760748.126685:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259776 00000100:00100000:0.0:1789760748.126801:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729355655083 time 729388471062 state 0x0 ref 1 00000100:00100000:0.0:1789760748.126819:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729388731478 time 729388471062 state 0x1 ref 1 00000100:00100000:0.0:1789760748.126864:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729388731478 time 729442167584 state 0x0 ref 1 00000100:00100000:0.0:1789760748.126880:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507259776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760748.127800:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20107264, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:1.0:1789760748.127817:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.127838:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760748.127843:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.127851:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.127867:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.186788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972229 is committed 00002000:00010000:0.0:1789760748.186802:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20107264 -> 20111360 00010000:00010000:0.0:1789760748.186825:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187055:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187070:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187078:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.187084:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187094:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187120:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.187134:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8276 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.187252:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507259776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60369us (62735us total) trans 4294972229 rc 0/0 00000100:00100000:0.0:1789760748.187266:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4939 00000100:00100000:0.0:1789760748.187279:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729388731478 time 729442167584 state 0x2 ref 0 00000100:00100000:1.0:1789760748.193195:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.194499:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507259904 00000100:00100000:2.0:1789760748.194528:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507259904 00000100:00100000:2.0:1789760748.194626:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729388731478 time 729442167584 state 0x0 ref 1 00000100:00100000:2.0:1789760748.194647:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729442428000 time 729442167584 state 0x1 ref 1 00000100:00100000:2.0:1789760748.194734:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729442428000 time 729509994470 state 0x0 ref 1 00000100:00100000:2.0:1789760748.194763:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.195003:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20111360, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.195014:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.195034:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.195040:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.195049:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.195067:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.237261:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972230 is committed 00002000:00010000:1.0:1789760748.237409:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20111360 -> 20115456 00010000:00010000:1.0:1789760748.237432:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237447:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237471:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237480:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760748.237485:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237497:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237522:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.237531:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b827d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.237661:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507259904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42895us (44474us total) trans 4294972230 rc 0/0 00000100:00100000:1.0:1789760748.237674:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4940 00000100:00100000:1.0:1789760748.237686:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729442428000 time 729509994470 state 0x2 ref 0 00000100:00100000:2.0:1789760748.244491:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.244781:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260032 00000100:00100000:0.0:1789760748.244795:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260032 00000100:00100000:0.0:1789760748.244944:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729442428000 time 729509994470 state 0x0 ref 1 00000100:00100000:0.0:1789760748.244969:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729510254886 time 729509994470 state 0x1 ref 1 00000100:00100000:0.0:1789760748.245005:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729510254886 time 729560307501 state 0x0 ref 1 00000100:00100000:0.0:1789760748.245021:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507260032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.245090:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20115456, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.245101:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.245121:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760748.245127:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.245135:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.245161:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.246546:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.246578:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.246673:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.246716:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.247522:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.247658:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260288 00000100:00100000:0.0:1789760748.247670:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260288 00000100:00100000:0.0:1789760748.247861:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260544 00000100:00100000:0.0:1789760748.247866:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260544 00000100:00100000:0.0:1789760748.247892:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260416 00000100:00100000:0.0:1789760748.247895:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260416 00000100:00100000:0.0:1789760748.247921:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 898 00000100:00100000:0.0:1789760748.247938:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507260288:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760748.248041:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2416:x1876699507260288:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 105us (1502us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.248049:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 898 00000100:00100000:0.0:1789760748.248072:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 899 00000100:00100000:0.0:1789760748.248079:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507260544:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760748.248109:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507260544:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 31us (1531us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.248115:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 899 00000100:00100000:0.0:1789760748.248126:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 900 00000100:00100000:0.0:1789760748.248132:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507260416:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760748.248156:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507260416:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 24us (1439us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.248162:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 900 00000100:00080000:0.0:1789760748.248181:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760748.248186:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760748.253309:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260160 00000100:00100000:1.0:1789760748.253331:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260160 00000100:00100000:1.0:1789760748.253403:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 470 00000100:00100000:1.0:1789760748.253415:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223497050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507260160:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760748.253517:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223497050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507260160:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 105us (6843us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.253530:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 470 00000100:00100000:1.0:1789760748.253583:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260672 00000100:00100000:1.0:1789760748.253588:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260672 00000100:00100000:1.0:1789760748.253631:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 901 00000100:00100000:1.0:1789760748.253638:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507260672:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760748.253671:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507260672:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 34us (6152us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.253678:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 901 00000100:00080000:1.0:1789760748.253717:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00010000:2.0:1789760748.303946:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20115456 -> 20119552 00010000:00010000:2.0:1789760748.303974:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.303989:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.304001:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.304009:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.304015:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.304026:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.304052:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.304060:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8284 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.304189:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507260032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59164us (59705us total) trans 4294972231 rc 0/0 00000100:00100000:2.0:1789760748.304202:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4941 00000100:00100000:2.0:1789760748.304214:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729510254886 time 729560307501 state 0x2 ref 0 00000001:00080000:1.0:1789760748.304895:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972231 is committed 00000100:00100000:0.0:1789760748.313314:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.313466:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260800 00000100:00100000:1.0:1789760748.313480:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260800 00000100:00100000:1.0:1789760748.313555:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729510254886 time 729560307501 state 0x0 ref 1 00000100:00100000:1.0:1789760748.313575:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729560567917 time 729560307501 state 0x1 ref 1 00000100:00100000:1.0:1789760748.313610:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729560567917 time 729628913299 state 0x0 ref 1 00000100:00100000:1.0:1789760748.313625:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507260800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760748.314125:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20119552, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:1.0:1789760748.314139:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.314160:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760748.314166:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.314174:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.314193:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.376359:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20119552 -> 20123648 00010000:00010000:2.0:1789760748.376385:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376400:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376410:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376419:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.376425:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376435:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376460:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.376474:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b828b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.376601:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507260800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62972us (63295us total) trans 4294972232 rc 0/0 00000100:00100000:2.0:1789760748.376614:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4942 00000100:00100000:2.0:1789760748.376627:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729560567917 time 729628913299 state 0x2 ref 0 00000001:00080000:3.0:1789760748.376730:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972232 is committed 00000100:00100000:3.0:1789760748.381333:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.381456:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507260928 00000100:00100000:1.0:1789760748.381468:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507260928 00000100:00100000:1.0:1789760748.381540:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729560567917 time 729628913299 state 0x0 ref 1 00000100:00100000:1.0:1789760748.381560:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729629173715 time 729628913299 state 0x1 ref 1 00000100:00100000:1.0:1789760748.381644:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729629173715 time 729696946062 state 0x0 ref 1 00000100:00100000:1.0:1789760748.381660:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507260928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760748.381790:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20123648, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:1.0:1789760748.381801:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.381820:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760748.381825:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.381833:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.381851:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.414108:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972233 is committed 00002000:00010000:1.0:1789760748.414216:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20123648 -> 20127744 00010000:00010000:1.0:1789760748.414237:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414253:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414264:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414273:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760748.414280:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414290:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414317:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.414325:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8292 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.414487:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507260928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32823us (33162us total) trans 4294972233 rc 0/0 00000100:00100000:1.0:1789760748.414514:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4943 00000100:00100000:1.0:1789760748.414527:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729629173715 time 729696946062 state 0x2 ref 0 00000100:00100000:1.0:1789760748.420588:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.420829:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261056 00000100:00100000:2.0:1789760748.420843:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261056 00000100:00100000:2.0:1789760748.420923:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729629173715 time 729696946062 state 0x0 ref 1 00000100:00100000:2.0:1789760748.420943:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729697206478 time 729696946062 state 0x1 ref 1 00000100:00100000:2.0:1789760748.420980:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729697206478 time 729736282887 state 0x0 ref 1 00000100:00100000:2.0:1789760748.420997:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.421068:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20127744, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.421079:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.421099:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.421103:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.421110:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.421133:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.445966:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760748.466339:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972234 is committed 00002000:00010000:3.0:1789760748.467002:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20127744 -> 20131840 00010000:00010000:3.0:1789760748.467033:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467049:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467059:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467067:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760748.467074:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467084:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467109:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467116:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8299 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.467254:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46247us (46666us total) trans 4294972234 rc 0/0 00000100:00100000:3.0:1789760748.467267:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4944 00000100:00100000:3.0:1789760748.467279:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729697206478 time 729736282887 state 0x2 ref 0 00000100:00100000:0.0:1789760748.473560:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.473658:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261184 00000100:00100000:2.0:1789760748.473672:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261184 00000100:00100000:2.0:1789760748.473792:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729697206478 time 729736282887 state 0x0 ref 1 00000100:00100000:2.0:1789760748.473811:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729736543303 time 729736282887 state 0x1 ref 1 00000100:00100000:2.0:1789760748.473846:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729736543303 time 729789148941 state 0x0 ref 1 00000100:00100000:2.0:1789760748.473861:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.473930:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20131840, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.473941:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.473961:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.473966:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.473975:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.473995:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.505745:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20131840 -> 20135936 00010000:00010000:1.0:1789760748.505906:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505923:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505933:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505941:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760748.505947:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505956:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505980:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.505988:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b82a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.506157:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507261184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32291us (32605us total) trans 4294972235 rc 0/0 00000100:00100000:1.0:1789760748.506171:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4945 00000100:00100000:1.0:1789760748.506184:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729736543303 time 729789148941 state 0x2 ref 0 00000001:00080000:3.0:1789760748.510161:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972235 is committed 00000100:00100000:1.0:1789760748.513889:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.514046:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261312 00000100:00100000:2.0:1789760748.514061:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261312 00000100:00100000:2.0:1789760748.514139:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729736543303 time 729789148941 state 0x0 ref 1 00000100:00100000:2.0:1789760748.514157:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729789409357 time 729789148941 state 0x1 ref 1 00000100:00100000:2.0:1789760748.514184:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729789409357 time 729829487433 state 0x0 ref 1 00000100:00100000:2.0:1789760748.514198:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507261312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.514265:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20135936, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.514276:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.514297:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.514302:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.514309:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.514328:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.552053:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972236 is committed 00002000:00010000:2.0:1789760748.552363:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20135936 -> 20140032 00010000:00010000:2.0:1789760748.552383:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552398:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552415:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552423:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.552428:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552488:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552513:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552520:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.552634:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507261312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38432us (38755us total) trans 4294972236 rc 0/0 00000100:00100000:2.0:1789760748.552646:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4946 00000100:00100000:2.0:1789760748.552658:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729789409357 time 729829487433 state 0x2 ref 0 00000100:00100000:2.0:1789760748.560037:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760748.560372:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261440 00000100:00100000:3.0:1789760748.560388:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261440 00000100:00100000:0.0:1789760748.560598:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729789409357 time 729829487433 state 0x0 ref 1 00000100:00100000:0.0:1789760748.560617:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729829747849 time 729829487433 state 0x1 ref 1 00000100:00100000:0.0:1789760748.560655:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729829747849 time 729875957994 state 0x0 ref 1 00000100:00100000:0.0:1789760748.560669:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.560785:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20140032, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.560797:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.560817:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760748.560823:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.560830:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.560849:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.607208:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972237 is committed 00002000:00010000:0.0:1789760748.607309:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20140032 -> 20144128 00010000:00010000:0.0:1789760748.607330:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607352:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607363:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607372:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.607377:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607389:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607414:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.607493:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.607846:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47172us (47817us total) trans 4294972237 rc 0/0 00000100:00100000:0.0:1789760748.607860:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4947 00000100:00100000:0.0:1789760748.607872:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729829747849 time 729875957994 state 0x2 ref 0 00000100:00100000:0.0:1789760748.617438:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760748.617560:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261568 00000100:00100000:1.0:1789760748.617574:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261568 00000100:00100000:1.0:1789760748.617662:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729829747849 time 729875957994 state 0x0 ref 1 00000100:00100000:1.0:1789760748.617682:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729876218410 time 729875957994 state 0x1 ref 1 00000100:00100000:1.0:1789760748.617777:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729876218410 time 729933079881 state 0x0 ref 1 00000100:00100000:1.0:1789760748.617792:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507261568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760748.617857:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20144128, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:1.0:1789760748.617868:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.617893:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760748.617898:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.617906:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.617926:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.665652:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20144128 -> 20148224 00010000:00010000:0.0:1789760748.665679:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665712:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665723:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665731:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.665746:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665763:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665790:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.665798:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b82b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.665926:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507261568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48126us (48500us total) trans 4294972238 rc 0/0 00000100:00100000:0.0:1789760748.665940:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4948 00000100:00100000:0.0:1789760748.665954:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729876218410 time 729933079881 state 0x2 ref 0 00000001:00080000:2.0:1789760748.666034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972238 is committed 00000100:00100000:1.0:1789760748.676550:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760748.677250:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261696 00000100:00100000:2.0:1789760748.677266:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261696 00000100:00100000:2.0:1789760748.677354:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729876218410 time 729933079881 state 0x0 ref 1 00000100:00100000:2.0:1789760748.677376:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729933340297 time 729933079881 state 0x1 ref 1 00000100:00100000:2.0:1789760748.677417:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729933340297 time 729992719446 state 0x0 ref 1 00000100:00100000:2.0:1789760748.677433:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507261696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760748.677504:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20148224, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760748.677515:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.677536:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760748.677542:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.677550:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.677568:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.731250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972239 is committed 00002000:00010000:0.0:1789760748.731339:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20148224 -> 20152320 00010000:00010000:0.0:1789760748.731361:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731379:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731390:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731398:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.731404:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731415:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731447:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.731456:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b82bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.731594:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507261696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54156us (55051us total) trans 4294972239 rc 0/0 00000100:00100000:0.0:1789760748.731606:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4949 00000100:00100000:0.0:1789760748.731619:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729933340297 time 729992719446 state 0x2 ref 0 00000100:00100000:3.0:1789760748.736526:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.736595:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261824 00000100:00100000:0.0:1789760748.736610:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261824 00000100:00100000:0.0:1789760748.736749:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729933340297 time 729992719446 state 0x0 ref 1 00000100:00100000:0.0:1789760748.736770:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729992979862 time 729992719446 state 0x1 ref 1 00000100:00100000:0.0:1789760748.736799:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729992979862 time 730052102365 state 0x0 ref 1 00000100:00100000:0.0:1789760748.736813:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507261824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.736878:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20152320, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.736888:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.736907:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.736911:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.736919:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.736935:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.764715:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538276224:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760748.765028:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760748.765222:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276224 00000100:00100000:2.0:1789760748.765233:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276224 00000100:00080000:3.0:1789760748.765330:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760748.765348:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1789760748.765360:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538276224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760748.765442:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760748.765635:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538276224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 277us (607us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760748.765643:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1789760748.765823:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbf8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538276352:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760748.765865:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.765907:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538276864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.765939:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.765960:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538277376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.765983:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.766005:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760748.766011:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108104e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538276480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766041:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.766064:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebe8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538276992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766088:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.766106:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebeb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538277504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766128:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.766160:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276352 00000100:00100000:0.0:1789760748.766164:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276352 00000100:00100000:0.0:1789760748.766205:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1789760748.766213:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538276352:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760748.766228:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760748.766256:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538276352:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766262:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1789760748.766279:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276864 00000100:00100000:0.0:1789760748.766282:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276864 00000100:00100000:0.0:1789760748.766305:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 902 00000100:00100000:0.0:1789760748.766311:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538276864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766339:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538276864:12345-0@lo:400:kworker.0 Request processed in 29us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766345:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 902 00000100:00100000:0.0:1789760748.766360:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277376 00000100:00100000:0.0:1789760748.766363:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277376 00000100:00100000:0.0:1789760748.766386:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277504 00000100:00100000:0.0:1789760748.766388:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277504 00000100:00100000:0.0:1789760748.766409:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789760748.766416:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538277376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766449:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538277376:12345-0@lo:400:kworker.0 Request processed in 34us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766455:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789760748.766477:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:0.0:1789760748.766483:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538277504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766514:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538277504:12345-0@lo:400:kworker.0 Request processed in 32us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766519:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:0.0:1789760748.766537:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276480 00000100:00100000:0.0:1789760748.766540:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276480 00000100:00100000:0.0:1789760748.766564:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760748.766570:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223494850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538276480:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766594:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223494850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538276480:12345-0@lo:400:kworker.0 Request processed in 25us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766600:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760748.766635:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276992 00000100:00100000:0.0:1789760748.766638:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276992 00000100:00100000:0.0:1789760748.766664:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789760748.766669:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538276992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766717:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538276992:12345-0@lo:400:kworker.0 Request processed in 48us (628us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.766723:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789760748.766779:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108104e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538276480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766797:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebe8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538276992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766807:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebeb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538277504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766815:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538277376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766823:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108104a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538276864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.766840:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbf8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538276352:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1789760748.767094:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760748.767100:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760748.767528:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108105180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538276608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.767580:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.767618:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538277120:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.767642:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.767670:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebe8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538277632:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.767722:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.767758:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760748.767764:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108106680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538276736:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.767927:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.767947:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebe9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538277248:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.767971:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.767993:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebe9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538277760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760748.768016:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760748.768105:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276608 00000100:00100000:1.0:1789760748.768113:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276608 00000100:00100000:1.0:1789760748.768155:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277120 00000100:00100000:1.0:1789760748.768158:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277120 00000100:00100000:1.0:1789760748.768180:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277632 00000100:00100000:1.0:1789760748.768183:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277632 00000100:00100000:1.0:1789760748.768200:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277248 00000100:00100000:1.0:1789760748.768203:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277248 00000100:00100000:1.0:1789760748.768216:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277760 00000100:00100000:1.0:1789760748.768218:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277760 00000100:00100000:1.0:1789760748.768249:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 903 00000100:00100000:1.0:1789760748.768260:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538276608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760748.768320:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538276224:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760748.768352:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 904 00000100:00100000:2.0:1789760748.768362:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538277120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760748.768428:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538277120:12345-0@lo:400:kworker.0 Request processed in 67us (784us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.768433:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538276608:12345-0@lo:400:kworker.0 Request processed in 168us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760748.768439:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 904 00000100:00100000:2.0:1789760748.768457:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 905 00000100:00100000:2.0:1789760748.768464:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538277632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760748.768495:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538277632:12345-0@lo:400:kworker.0 Request processed in 31us (770us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.768501:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 903 00000100:00100000:2.0:1789760748.768508:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 905 00000100:00100000:1.0:1789760748.768521:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 906 00000100:00100000:2.0:1789760748.768529:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 907 00000100:00100000:1.0:1789760748.768534:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538277248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760748.768541:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538277760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768571:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538277248:12345-0@lo:400:kworker.0 Request processed in 41us (598us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.768577:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 906 00000100:00100000:1.0:1789760748.768601:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538276736 00000100:00100000:1.0:1789760748.768605:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538276736 00000100:00100000:1.0:1789760748.768638:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1789760748.768639:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538277760:12345-0@lo:400:kworker.0 Request processed in 99us (619us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.768648:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538276736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760748.768652:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 907 00000100:00100000:1.0:1789760748.768817:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538276736:12345-0@lo:400:kworker.0 Request processed in 172us (889us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760748.768825:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:1.0:1789760748.768854:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108106680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538276736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768870:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebe9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538277248:0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768881:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebe9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538277760:0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768892:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108105180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538276608:0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768900:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538277120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760748.768906:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebe8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538277632:0@lo:400:kworker.0 00000100:00080000:0.0:1789760748.769296:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760748.769301:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760748.783725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972240 is committed 00002000:00010000:0.0:1789760748.783760:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20152320 -> 20156416 00010000:00010000:0.0:1789760748.783780:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783795:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783804:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783812:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.783817:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783828:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783854:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.783863:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.783974:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507261824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47157us (47458us total) trans 4294972240 rc 0/0 00000100:00100000:0.0:1789760748.783985:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4950 00000100:00100000:0.0:1789760748.783997:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729992979862 time 730052102365 state 0x2 ref 0 00000100:00100000:2.0:1789760748.787311:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.787402:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507261952 00000100:00100000:0.0:1789760748.787413:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507261952 00000100:00100000:0.0:1789760748.787485:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 729992979862 time 730052102365 state 0x0 ref 1 00000100:00100000:0.0:1789760748.787583:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730052362781 time 730052102365 state 0x1 ref 1 00000100:00100000:0.0:1789760748.787619:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730052362781 time 730102922079 state 0x0 ref 1 00000100:00100000:0.0:1789760748.787631:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.787892:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20156416, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.787904:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.787924:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.787930:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.787938:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.787956:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.827250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972241 is committed 00002000:00010000:0.0:1789760748.827371:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20156416 -> 20160512 00010000:00010000:0.0:1789760748.827392:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827408:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827418:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827425:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.827431:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827542:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827571:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.827587:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.827887:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507261952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40242us (40572us total) trans 4294972241 rc 0/0 00000100:00100000:0.0:1789760748.827901:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4951 00000100:00100000:0.0:1789760748.827914:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730052362781 time 730102922079 state 0x2 ref 0 00000100:00100000:1.0:1789760748.831452:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.831785:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262080 00000100:00100000:0.0:1789760748.831799:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262080 00000100:00100000:0.0:1789760748.831874:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730052362781 time 730102922079 state 0x0 ref 1 00000100:00100000:0.0:1789760748.831903:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730103182495 time 730102922079 state 0x1 ref 1 00000100:00100000:0.0:1789760748.831931:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730103182495 time 730147233704 state 0x0 ref 1 00000100:00100000:0.0:1789760748.831948:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507262080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.832027:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20160512, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.832040:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.832063:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.832069:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.832077:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.832101:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.875504:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20160512 -> 20164608 00010000:00010000:0.0:1789760748.875529:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875543:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875552:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875560:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760748.875565:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875574:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875595:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.875604:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.876155:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507262080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44205us (44711us total) trans 4294972242 rc 0/0 00000100:00100000:0.0:1789760748.876168:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4952 00000100:00100000:0.0:1789760748.876178:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730103182495 time 730147233704 state 0x2 ref 0 00000001:00080000:0.0:1789760748.876214:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972242 is committed 00000100:00100000:3.0:1789760748.882203:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.884678:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262208 00000100:00100000:0.0:1789760748.884708:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262208 00000100:00100000:0.0:1789760748.884945:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730103182495 time 730147233704 state 0x0 ref 1 00000100:00100000:0.0:1789760748.884962:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730147494120 time 730147233704 state 0x1 ref 1 00000100:00100000:0.0:1789760748.885041:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730147494120 time 730200343337 state 0x0 ref 1 00000100:00100000:0.0:1789760748.885056:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.885110:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20164608, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.885120:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.885136:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.885142:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.885149:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.885164:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.916898:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972243 is committed 00002000:00010000:2.0:1789760748.916989:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20164608 -> 20168704 00010000:00010000:2.0:1789760748.917005:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917019:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917029:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917036:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.917040:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917050:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917068:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.917077:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b82d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.917212:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32154us (35014us total) trans 4294972243 rc 0/0 00000100:00100000:2.0:1789760748.917223:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4953 00000100:00100000:2.0:1789760748.917233:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730147494120 time 730200343337 state 0x2 ref 0 00000100:00100000:2.0:1789760748.925377:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.925470:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262336 00000100:00100000:0.0:1789760748.925480:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262336 00000100:00100000:0.0:1789760748.925534:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730147494120 time 730200343337 state 0x0 ref 1 00000100:00100000:0.0:1789760748.925550:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730200603753 time 730200343337 state 0x1 ref 1 00000100:00100000:0.0:1789760748.925582:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730200603753 time 730240884976 state 0x0 ref 1 00000100:00100000:0.0:1789760748.925594:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507262336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.925644:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20168704, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.925653:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.925669:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.925673:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.925680:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.925887:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.975383:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972244 is committed 00002000:00010000:2.0:1789760748.975492:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20168704 -> 20172800 00010000:00010000:2.0:1789760748.975507:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975520:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975528:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975540:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760748.975545:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975553:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975573:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.975581:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b82df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.975747:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507262336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50142us (50367us total) trans 4294972244 rc 0/0 00000100:00100000:2.0:1789760748.975758:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4954 00000100:00100000:2.0:1789760748.975776:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730200603753 time 730240884976 state 0x2 ref 0 00000100:00100000:1.0:1789760748.979878:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760748.980259:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262464 00000100:00100000:0.0:1789760748.980270:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262464 00000100:00100000:0.0:1789760748.980323:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730200603753 time 730240884976 state 0x0 ref 1 00000100:00100000:0.0:1789760748.980338:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730241145392 time 730240884976 state 0x1 ref 1 00000100:00100000:0.0:1789760748.980368:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730241145392 time 730295670560 state 0x0 ref 1 00000100:00100000:0.0:1789760748.980380:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760748.980430:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20172800, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:0.0:1789760748.980439:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.980455:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760748.980459:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.980465:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.980479:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.015805:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972245 is committed 00002000:00010000:2.0:1789760749.015899:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20172800 -> 20176896 00010000:00010000:2.0:1789760749.015914:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.015927:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.015942:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.015949:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760749.015954:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.015973:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.015993:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.016002:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b82e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.016176:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35792us (36303us total) trans 4294972245 rc 0/0 00000100:00100000:2.0:1789760749.016187:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4955 00000100:00100000:2.0:1789760749.016197:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730241145392 time 730295670560 state 0x2 ref 0 00000100:00100000:1.0:1789760749.021812:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760749.021944:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262592 00000100:00100000:2.0:1789760749.021954:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262592 00000100:00100000:2.0:1789760749.022123:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730241145392 time 730295670560 state 0x0 ref 1 00000100:00100000:2.0:1789760749.022138:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730295930976 time 730295670560 state 0x1 ref 1 00000100:00100000:2.0:1789760749.022168:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730295930976 time 730337471189 state 0x0 ref 1 00000100:00100000:2.0:1789760749.022182:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760749.022228:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20176896, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa007 00010000:00010000:2.0:1789760749.022237:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.022252:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760749.022257:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.022263:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.022277:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760749.023937:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760749.054612:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972246 is committed 00002000:00010000:0.0:1789760749.054676:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20176896 -> 20180992 00002000:00010000:0.0:1789760749.054680:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:0.0:1789760749.054682:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:0.0:1789760749.054713:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054727:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054754:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054761:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.054766:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054783:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054802:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.054811:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b82ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.054907:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32723us (33098us total) trans 4294972246 rc 0/0 00000100:00100000:0.0:1789760749.054918:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4956 00000100:00100000:0.0:1789760749.054928:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730295930976 time 730337471189 state 0x2 ref 0 00000100:00100000:0.0:1789760749.058181:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760749.058260:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262720 00000100:00100000:2.0:1789760749.058271:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262720 00000100:00100000:2.0:1789760749.058324:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730295930976 time 730337471189 state 0x0 ref 1 00000100:00100000:2.0:1789760749.058339:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730337731605 time 730337471189 state 0x1 ref 1 00000100:00100000:2.0:1789760749.058368:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730337731605 time 730373671589 state 0x0 ref 1 00000100:00100000:2.0:1789760749.058380:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760749.058428:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20180992, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:2.0:1789760749.058439:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.058455:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760749.058459:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.058465:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.058479:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.103884:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20180992 -> 20185088 00000001:00080000:3.0:1789760749.103887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972247 is committed 00010000:00010000:0.0:1789760749.103905:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103924:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103934:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103941:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.103946:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103968:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103990:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.103998:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b82f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.104093:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507262720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45708us (45914us total) trans 4294972247 rc 0/0 00000100:00100000:0.0:1789760749.104104:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4957 00000100:00100000:0.0:1789760749.104114:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730337731605 time 730373671589 state 0x2 ref 0 00000100:00100000:2.0:1789760749.109380:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.109466:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262848 00000100:00100000:1.0:1789760749.109475:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262848 00000100:00100000:1.0:1789760749.109536:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730337731605 time 730373671589 state 0x0 ref 1 00000100:00100000:1.0:1789760749.109551:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730373932005 time 730373671589 state 0x1 ref 1 00000100:00100000:1.0:1789760749.109613:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730373932005 time 730424916253 state 0x0 ref 1 00000100:00100000:1.0:1789760749.109626:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507262848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.109675:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20185088, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.109683:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.109716:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760749.109744:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.109753:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.109768:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.156185:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972248 is committed 00002000:00010000:0.0:1789760749.156297:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20185088 -> 20189184 00010000:00010000:0.0:1789760749.156312:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156325:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156335:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156342:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.156347:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156355:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156377:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.156384:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b82fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.156514:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507262848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46885us (47140us total) trans 4294972248 rc 0/0 00000100:00100000:0.0:1789760749.156525:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4958 00000100:00100000:0.0:1789760749.156534:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730373932005 time 730424916253 state 0x2 ref 0 00000100:00100000:1.0:1789760749.166348:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.166437:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507262976 00000100:00100000:0.0:1789760749.166447:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507262976 00000100:00100000:0.0:1789760749.166495:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730373932005 time 730424916253 state 0x0 ref 1 00000100:00100000:0.0:1789760749.166509:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730425176669 time 730424916253 state 0x1 ref 1 00000100:00100000:0.0:1789760749.166536:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730425176669 time 730481839411 state 0x0 ref 1 00000100:00100000:0.0:1789760749.166547:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507262976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.166603:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20189184, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.166612:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.166627:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760749.166631:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.166637:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.166650:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.221165:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20189184 -> 20193280 00010000:00010000:3.0:1789760749.221187:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221201:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221217:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221225:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760749.221230:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221240:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221261:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.221270:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8302 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.221361:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507262976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54809us (55017us total) trans 4294972249 rc 0/0 00000100:00100000:3.0:1789760749.221372:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4959 00000100:00100000:3.0:1789760749.221382:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730425176669 time 730481839411 state 0x2 ref 0 00000001:00080000:0.0:1789760749.221955:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972249 is committed 00000100:00100000:2.0:1789760749.226397:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.226469:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263104 00000100:00100000:0.0:1789760749.226478:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263104 00000100:00100000:0.0:1789760749.226525:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730425176669 time 730481839411 state 0x0 ref 1 00000100:00100000:0.0:1789760749.226541:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730482099827 time 730481839411 state 0x1 ref 1 00000100:00100000:0.0:1789760749.226574:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730482099827 time 730541877511 state 0x0 ref 1 00000100:00100000:0.0:1789760749.226593:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507263104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.226640:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20193280, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.226662:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.226678:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760749.226683:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.226690:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.226755:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.266341:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972250 is committed 00002000:00010000:2.0:1789760749.266398:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20193280 -> 20197376 00010000:00010000:2.0:1789760749.266424:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266437:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266446:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266454:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760749.266459:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266474:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266493:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.266501:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8309 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.266591:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507263104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39997us (40196us total) trans 4294972250 rc 0/0 00000100:00100000:2.0:1789760749.266601:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4960 00000100:00100000:2.0:1789760749.266610:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730482099827 time 730541877511 state 0x2 ref 0 00000100:00100000:2.0:1789760749.270126:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.270218:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263232 00000100:00100000:0.0:1789760749.270229:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263232 00000100:00100000:0.0:1789760749.270287:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730482099827 time 730541877511 state 0x0 ref 1 00000100:00100000:0.0:1789760749.270302:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730542137927 time 730541877511 state 0x1 ref 1 00000100:00100000:0.0:1789760749.270333:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730542137927 time 730585636123 state 0x0 ref 1 00000100:00100000:0.0:1789760749.270345:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507263232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.270394:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20197376, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.270403:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.270416:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760749.270421:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.270427:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.270441:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.305606:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20197376 -> 20201472 00010000:00010000:0.0:1789760749.305625:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305637:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305646:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305653:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.305658:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305667:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305688:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.305715:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8310 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.305859:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507263232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35510us (35736us total) trans 4294972251 rc 0/0 00000100:00100000:0.0:1789760749.305870:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4961 00000100:00100000:0.0:1789760749.305880:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730542137927 time 730585636123 state 0x2 ref 0 00000001:00080000:2.0:1789760749.306007:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972251 is committed 00000100:00100000:1.0:1789760749.321892:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760749.322117:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263360 00000100:00100000:3.0:1789760749.322127:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263360 00000100:00100000:3.0:1789760749.322179:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730542137927 time 730585636123 state 0x0 ref 1 00000100:00100000:3.0:1789760749.322195:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730585896539 time 730585636123 state 0x1 ref 1 00000100:00100000:3.0:1789760749.322226:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730585896539 time 730637530056 state 0x0 ref 1 00000100:00100000:3.0:1789760749.322239:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507263360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760749.322286:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20201472, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:3.0:1789760749.322296:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.322311:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760749.322316:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.322322:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.322338:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.404426:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972252 is committed 00002000:00010000:1.0:1789760749.404527:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20201472 -> 20205568 00010000:00010000:1.0:1789760749.404542:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404554:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404563:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404570:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.404575:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404583:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404602:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.404615:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8317 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.404758:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507263360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82517us (82872us total) trans 4294972252 rc 0/0 00000100:00100000:1.0:1789760749.404776:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4962 00000100:00100000:1.0:1789760749.404786:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730585896539 time 730637530056 state 0x2 ref 0 00000100:00100000:1.0:1789760749.407634:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760749.407743:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263488 00000100:00100000:2.0:1789760749.407753:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263488 00000100:00100000:2.0:1789760749.407816:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730585896539 time 730637530056 state 0x0 ref 1 00000100:00100000:2.0:1789760749.407831:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730637790472 time 730637530056 state 0x1 ref 1 00000100:00100000:2.0:1789760749.407901:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730637790472 time 730723203559 state 0x0 ref 1 00000100:00100000:2.0:1789760749.407913:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507263488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760749.407964:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20205568, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:2.0:1789760749.407973:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.407989:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760749.407993:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.408000:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.408014:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.447589:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972253 is committed 00002000:00010000:0.0:1789760749.447666:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20205568 -> 20209664 00010000:00010000:0.0:1789760749.447680:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449239:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449253:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449261:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.449265:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449281:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449302:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.449311:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b831e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.449472:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507263488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41554us (41840us total) trans 4294972253 rc 0/0 00000100:00100000:1.0:1789760749.449483:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4963 00000100:00100000:1.0:1789760749.449493:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730637790472 time 730723203559 state 0x2 ref 0 00000100:00100000:2.0:1789760749.453369:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.453445:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263616 00000100:00100000:0.0:1789760749.453456:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263616 00000100:00100000:0.0:1789760749.453512:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730637790472 time 730723203559 state 0x0 ref 1 00000100:00100000:0.0:1789760749.453527:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730723463975 time 730723203559 state 0x1 ref 1 00000100:00100000:0.0:1789760749.453557:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730723463975 time 730768860826 state 0x0 ref 1 00000100:00100000:0.0:1789760749.453582:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507263616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.453630:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20209664, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.453639:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.453654:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760749.453659:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.453666:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.453682:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.468975:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760749.481485:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20209664 -> 20213760 00010000:00010000:0.0:1789760749.481508:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481521:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481530:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481538:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.481543:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481552:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481572:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.481580:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8325 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.481665:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507263616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28093us (28300us total) trans 4294972254 rc 0/0 00000100:00100000:0.0:1789760749.481676:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4964 00000100:00100000:0.0:1789760749.481684:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730723463975 time 730768860826 state 0x2 ref 0 00000001:00080000:1.0:1789760749.482463:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972254 is committed 00000100:00100000:3.0:1789760749.485275:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.485403:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263744 00000100:00100000:1.0:1789760749.485413:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263744 00000100:00100000:1.0:1789760749.485468:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730723463975 time 730768860826 state 0x0 ref 1 00000100:00100000:1.0:1789760749.485484:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730769121242 time 730768860826 state 0x1 ref 1 00000100:00100000:1.0:1789760749.485551:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730769121242 time 730800853802 state 0x0 ref 1 00000100:00100000:1.0:1789760749.485564:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507263744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.485614:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20213760, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.485628:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.485646:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760749.485651:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.485658:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.485674:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.514895:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972255 is committed 00002000:00010000:1.0:1789760749.515052:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20213760 -> 20217856 00010000:00010000:1.0:1789760749.515068:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515081:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515091:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515098:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.515103:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515111:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515132:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.515140:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b832c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.515261:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507263744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29694us (29991us total) trans 4294972255 rc 0/0 00000100:00100000:1.0:1789760749.515272:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4965 00000100:00100000:1.0:1789760749.515282:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730769121242 time 730800853802 state 0x2 ref 0 00000100:00100000:0.0:1789760749.524249:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.524363:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507263872 00000100:00100000:1.0:1789760749.524374:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507263872 00000100:00100000:1.0:1789760749.524428:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730769121242 time 730800853802 state 0x0 ref 1 00000100:00100000:1.0:1789760749.524444:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730801114218 time 730800853802 state 0x1 ref 1 00000100:00100000:1.0:1789760749.524475:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730801114218 time 730839778016 state 0x0 ref 1 00000100:00100000:1.0:1789760749.524488:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507263872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.524542:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20217856, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.524551:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.524568:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760749.524573:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.524579:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.524593:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.551585:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20217856 -> 20221952 00010000:00010000:3.0:1789760749.551603:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551616:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551626:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551638:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760749.551643:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551652:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551670:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.551678:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8333 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.551802:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507263872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27312us (27559us total) trans 4294972256 rc 0/0 00000100:00100000:3.0:1789760749.551812:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4966 00000100:00100000:3.0:1789760749.551822:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730801114218 time 730839778016 state 0x2 ref 0 00000001:00080000:2.0:1789760749.552406:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972256 is committed 00000100:00100000:2.0:1789760749.555967:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.556067:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264000 00000100:00100000:0.0:1789760749.556088:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264000 00000100:00100000:0.0:1789760749.556145:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730801114218 time 730839778016 state 0x0 ref 1 00000100:00100000:0.0:1789760749.556162:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730840038432 time 730839778016 state 0x1 ref 1 00000100:00100000:0.0:1789760749.556199:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730840038432 time 730871501842 state 0x0 ref 1 00000100:00100000:0.0:1789760749.556212:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507264000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.556260:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20221952, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.556270:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.556284:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760749.556289:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.556295:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.556310:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.586260:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20221952 -> 20226048 00010000:00010000:0.0:1789760749.586278:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586291:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586300:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586307:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.586311:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586321:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586341:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586348:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b833a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.586447:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507264000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30229us (30480us total) trans 4294972257 rc 0/0 00000100:00100000:0.0:1789760749.586461:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4967 00000100:00100000:0.0:1789760749.586471:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730840038432 time 730871501842 state 0x2 ref 0 00000001:00080000:3.0:1789760749.587117:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972257 is committed 00000100:00100000:0.0:1789760749.590071:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.590188:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264128 00000100:00100000:1.0:1789760749.590197:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264128 00000100:00100000:1.0:1789760749.590248:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730840038432 time 730871501842 state 0x0 ref 1 00000100:00100000:1.0:1789760749.590263:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730871762258 time 730871501842 state 0x1 ref 1 00000100:00100000:1.0:1789760749.590325:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730871762258 time 730905627350 state 0x0 ref 1 00000100:00100000:1.0:1789760749.590337:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.590386:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20226048, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.590394:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.591873:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760749.591882:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.591890:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.591909:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.655912:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972258 is committed 00002000:00010000:1.0:1789760749.655998:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20226048 -> 20230144 00010000:00010000:1.0:1789760749.656011:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656024:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656032:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656039:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.656044:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656052:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656069:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.656075:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8341 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.656182:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65842us (66115us total) trans 4294972258 rc 0/0 00000100:00100000:1.0:1789760749.656196:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4968 00000100:00100000:1.0:1789760749.656205:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730871762258 time 730905627350 state 0x2 ref 0 00000100:00100000:2.0:1789760749.661243:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760749.661596:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264256 00000100:00100000:3.0:1789760749.661606:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264256 00000100:00100000:3.0:1789760749.661658:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730871762258 time 730905627350 state 0x0 ref 1 00000100:00100000:3.0:1789760749.661672:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730905887766 time 730905627350 state 0x1 ref 1 00000100:00100000:3.0:1789760749.661799:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730905887766 time 730977101445 state 0x0 ref 1 00000100:00100000:3.0:1789760749.661812:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760749.661858:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20230144, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:3.0:1789760749.661868:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.661883:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760749.661888:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.661894:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.661909:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.690034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972259 is committed 00002000:00010000:0.0:1789760749.690118:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20230144 -> 20234240 00010000:00010000:0.0:1789760749.690134:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690147:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690156:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690164:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.690169:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690178:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690198:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690206:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8348 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.690334:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28519us (29095us total) trans 4294972259 rc 0/0 00000100:00100000:0.0:1789760749.690345:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4969 00000100:00100000:0.0:1789760749.690354:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730905887766 time 730977101445 state 0x2 ref 0 00000100:00100000:3.0:1789760749.694479:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.694555:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264384 00000100:00100000:0.0:1789760749.694564:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264384 00000100:00100000:0.0:1789760749.694613:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730905887766 time 730977101445 state 0x0 ref 1 00000100:00100000:0.0:1789760749.694627:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730977361861 time 730977101445 state 0x1 ref 1 00000100:00100000:0.0:1789760749.694656:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730977361861 time 731009960285 state 0x0 ref 1 00000100:00100000:0.0:1789760749.694667:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.694751:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20234240, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.694760:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.694840:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760749.694846:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.694852:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.694867:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.745070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972260 is committed 00002000:00010000:0.0:1789760749.745179:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20234240 -> 20238336 00010000:00010000:0.0:1789760749.745195:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745207:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745217:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745225:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760749.745229:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745239:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745258:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.745266:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b834f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.745460:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50721us (50916us total) trans 4294972260 rc 0/0 00000100:00100000:0.0:1789760749.745477:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4970 00000100:00100000:0.0:1789760749.745487:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730977361861 time 731009960285 state 0x2 ref 0 00000100:00100000:3.0:1789760749.750319:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.750412:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264512 00000100:00100000:1.0:1789760749.750422:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264512 00000100:00100000:1.0:1789760749.750477:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 730977361861 time 731009960285 state 0x0 ref 1 00000100:00100000:1.0:1789760749.750492:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731010220701 time 731009960285 state 0x1 ref 1 00000100:00100000:1.0:1789760749.750525:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731010220701 time 731065828397 state 0x0 ref 1 00000100:00100000:1.0:1789760749.750538:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.750584:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20238336, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.750593:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.750610:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760749.750614:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.750620:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.750643:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.782758:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20238336 -> 20242432 00010000:00010000:2.0:1789760749.782787:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782802:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782811:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782826:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760749.782831:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782840:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782858:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.782865:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8356 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.783066:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507264512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32525us (32751us total) trans 4294972261 rc 0/0 00000100:00100000:2.0:1789760749.783076:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4971 00000100:00100000:2.0:1789760749.783086:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731010220701 time 731065828397 state 0x2 ref 0 00000001:00080000:0.0:1789760749.783440:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972261 is committed 00000100:00100000:2.0:1789760749.787030:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.787116:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264640 00000100:00100000:0.0:1789760749.787126:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264640 00000100:00100000:0.0:1789760749.787189:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731010220701 time 731065828397 state 0x0 ref 1 00000100:00100000:0.0:1789760749.787204:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731066088813 time 731065828397 state 0x1 ref 1 00000100:00100000:0.0:1789760749.787237:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731066088813 time 731102541076 state 0x0 ref 1 00000100:00100000:0.0:1789760749.787249:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.787299:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20242432, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.787308:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.787324:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760749.787328:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.787334:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.787348:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.787810:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108106680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538277888:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760749.787868:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760749.787883:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538277888 00000100:00100000:3.0:1789760749.787889:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538277888 00000100:00100000:3.0:1789760749.787921:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:3.0:1789760749.787929:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538277888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760749.787942:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3837788160 avail 3812622336 00000100:00100000:3.0:1789760749.787991:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538277888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (122us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.787998:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1789760749.788037:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108106680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538277888:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760749.788851:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760749.788862:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebeb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538278016:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760749.788968:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760749.789244:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278016 00000100:00100000:2.0:1789760749.789250:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278016 00000100:00100000:2.0:1789760749.789280:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789760749.789287:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538278016:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760749.789301:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3837788160 avail 3812622336 00000100:00100000:2.0:1789760749.789342:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538278016:12345-0@lo:13:osp-pre-0-1.0 Request processed in 55us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760749.789349:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:1.0:1789760749.789612:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebeb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538278016:0@lo:13:osp-pre-0-1.0 00000001:00080000:0.0:1789760749.832372:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972262 is committed 00002000:00010000:1.0:1789760749.832882:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20242432 -> 20246528 00010000:00010000:1.0:1789760749.832899:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832912:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832921:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832929:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.832934:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832943:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832965:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.832973:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b835d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.833275:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507264640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46019us (46246us total) trans 4294972262 rc 0/0 00000100:00100000:1.0:1789760749.833286:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4972 00000100:00100000:1.0:1789760749.833296:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731066088813 time 731102541076 state 0x2 ref 0 00000100:00100000:3.0:1789760749.845512:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760749.848143:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264768 00000100:00100000:0.0:1789760749.848158:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264768 00000100:00100000:0.0:1789760749.848233:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731066088813 time 731102541076 state 0x0 ref 1 00000100:00100000:0.0:1789760749.848248:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731102801492 time 731102541076 state 0x1 ref 1 00000100:00100000:0.0:1789760749.848285:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731102801492 time 731163587221 state 0x0 ref 1 00000100:00100000:0.0:1789760749.848299:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760749.848345:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20246528, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:0.0:1789760749.848354:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.848368:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760749.848373:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.848379:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.848393:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.901957:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972263 is committed 00002000:00010000:1.0:1789760749.902048:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20246528 -> 20250624 00010000:00010000:1.0:1789760749.902064:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902087:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902096:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902102:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.902107:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902115:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902134:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.902141:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8364 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.903098:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53936us (56728us total) trans 4294972263 rc 0/0 00000100:00100000:1.0:1789760749.903111:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4973 00000100:00100000:1.0:1789760749.903121:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731102801492 time 731163587221 state 0x2 ref 0 00000100:00100000:3.0:1789760749.905461:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760749.905555:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507264896 00000100:00100000:2.0:1789760749.905566:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507264896 00000100:00100000:2.0:1789760749.905623:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731102801492 time 731163587221 state 0x0 ref 1 00000100:00100000:2.0:1789760749.905639:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731163847637 time 731163587221 state 0x1 ref 1 00000100:00100000:3.0:1789760749.905934:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731163847637 time 731221233914 state 0x0 ref 1 00000100:00100000:3.0:1789760749.905951:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507264896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760749.906001:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20250624, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:3.0:1789760749.906011:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.906025:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760749.906030:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.906036:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.906050:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.940360:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20250624 -> 20254720 00010000:00010000:3.0:1789760749.940379:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940393:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940403:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940416:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760749.940421:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940430:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940450:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.940458:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b836b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.940547:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507264896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34596us (35090us total) trans 4294972264 rc 0/0 00000100:00100000:3.0:1789760749.940557:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4974 00000100:00100000:3.0:1789760749.940567:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731163847637 time 731221233914 state 0x2 ref 0 00000001:00080000:1.0:1789760749.941113:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972264 is committed 00000100:00100000:0.0:1789760749.944108:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.944203:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265024 00000100:00100000:1.0:1789760749.944212:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265024 00000100:00100000:1.0:1789760749.944312:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731163847637 time 731221233914 state 0x0 ref 1 00000100:00100000:1.0:1789760749.944324:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731221494330 time 731221233914 state 0x1 ref 1 00000100:00100000:1.0:1789760749.944347:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731221494330 time 731259651934 state 0x0 ref 1 00000100:00100000:1.0:1789760749.944356:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507265024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.944405:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20254720, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.944413:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.944428:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760749.944433:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.944440:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.944455:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.981927:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972265 is committed 00002000:00010000:1.0:1789760749.982435:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20254720 -> 20258816 00010000:00010000:1.0:1789760749.982520:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982534:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982543:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982550:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760749.982554:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982562:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982582:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982591:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8372 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.982798:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507265024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38437us (38693us total) trans 4294972265 rc 0/0 00000100:00100000:1.0:1789760749.982812:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4975 00000100:00100000:1.0:1789760749.982821:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731221494330 time 731259651934 state 0x2 ref 0 00000100:00100000:0.0:1789760749.987150:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760749.987314:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265152 00000100:00100000:1.0:1789760749.987325:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265152 00000100:00100000:1.0:1789760749.987375:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731221494330 time 731259651934 state 0x0 ref 1 00000100:00100000:1.0:1789760749.987389:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731259912350 time 731259651934 state 0x1 ref 1 00000100:00100000:1.0:1789760749.987420:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731259912350 time 731302723236 state 0x0 ref 1 00000100:00100000:1.0:1789760749.987431:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760749.987486:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20258816, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa007 00010000:00010000:1.0:1789760749.987494:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.987511:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760749.987515:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.987521:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.987534:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.032627:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972266 is committed 00002000:00010000:2.0:1789760750.032824:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20258816 -> 20262912 00002000:00010000:2.0:1789760750.032829:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:2.0:1789760750.032832:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:2.0:1789760750.032845:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032858:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032868:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032876:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.032881:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032891:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032910:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.032918:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8379 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.033014:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45579us (45869us total) trans 4294972266 rc 0/0 00000100:00100000:2.0:1789760750.033025:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4976 00000100:00100000:2.0:1789760750.033036:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731259912350 time 731302723236 state 0x2 ref 0 00000100:00100000:1.0:1789760750.037515:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760750.037668:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265280 00000100:00100000:3.0:1789760750.037679:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265280 00000100:00100000:3.0:1789760750.037791:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731259912350 time 731302723236 state 0x0 ref 1 00000100:00100000:3.0:1789760750.037807:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731302983652 time 731302723236 state 0x1 ref 1 00000100:00100000:3.0:1789760750.037833:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731302983652 time 731353136902 state 0x0 ref 1 00000100:00100000:3.0:1789760750.037845:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760750.037892:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20262912, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:3.0:1789760750.037901:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.037916:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760750.037921:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.037928:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.037945:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.079737:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972267 is committed 00002000:00010000:1.0:1789760750.080002:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20262912 -> 20267008 00010000:00010000:1.0:1789760750.080017:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080030:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080039:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080046:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760750.080051:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080058:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080078:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.080085:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8380 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.080306:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42457us (42795us total) trans 4294972267 rc 0/0 00000100:00100000:1.0:1789760750.080316:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4977 00000100:00100000:1.0:1789760750.080327:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731302983652 time 731353136902 state 0x2 ref 0 00000100:00100000:3.0:1789760750.084434:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760750.084905:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265408 00000100:00100000:1.0:1789760750.084915:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265408 00000100:00100000:1.0:1789760750.084971:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731302983652 time 731353136902 state 0x0 ref 1 00000100:00100000:1.0:1789760750.084986:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731353397318 time 731353136902 state 0x1 ref 1 00000100:00100000:1.0:1789760750.085010:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731353397318 time 731400313354 state 0x0 ref 1 00000100:00100000:1.0:1789760750.085022:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760750.085073:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20267008, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:1.0:1789760750.085082:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.085097:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760750.085101:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.085108:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.085122:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.117516:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20267008 -> 20271104 00010000:00010000:2.0:1789760750.117533:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117546:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117555:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117562:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.117568:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117577:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117603:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.117612:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8387 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.117896:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972268 is committed 00000100:00100000:2.0:1789760750.118121:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507265408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33096us (33692us total) trans 4294972268 rc 0/0 00000100:00100000:2.0:1789760750.118133:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4978 00000100:00100000:2.0:1789760750.118143:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731353397318 time 731400313354 state 0x2 ref 0 00000100:00100000:0.0:1789760750.125625:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.125753:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265536 00000100:00100000:2.0:1789760750.125770:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265536 00000100:00100000:2.0:1789760750.125823:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731353397318 time 731400313354 state 0x0 ref 1 00000100:00100000:2.0:1789760750.125839:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731400573770 time 731400313354 state 0x1 ref 1 00000100:00100000:2.0:1789760750.125868:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731400573770 time 731441170999 state 0x0 ref 1 00000100:00100000:2.0:1789760750.125880:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507265536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.125927:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20271104, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.125936:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.125952:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760750.125957:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.125964:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.125979:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.162151:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972269 is committed 00002000:00010000:1.0:1789760750.162263:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20271104 -> 20275200 00010000:00010000:1.0:1789760750.162279:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162291:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162302:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162310:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760750.162315:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162323:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162342:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.162349:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b838e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.162442:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507265536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36560us (36821us total) trans 4294972269 rc 0/0 00000100:00100000:1.0:1789760750.162517:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4979 00000100:00100000:1.0:1789760750.162527:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731400573770 time 731441170999 state 0x2 ref 0 00000100:00100000:1.0:1789760750.171255:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760750.171382:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265664 00000100:00100000:3.0:1789760750.171392:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265664 00000100:00100000:3.0:1789760750.171443:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731400573770 time 731441170999 state 0x0 ref 1 00000100:00100000:3.0:1789760750.171520:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731441431415 time 731441170999 state 0x1 ref 1 00000100:00100000:3.0:1789760750.171590:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731441431415 time 731486893275 state 0x0 ref 1 00000100:00100000:3.0:1789760750.171606:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507265664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760750.171664:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20275200, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:3.0:1789760750.171673:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.171688:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760750.171738:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.171745:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.171761:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.218789:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972270 is committed 00002000:00010000:3.0:1789760750.218861:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20275200 -> 20279296 00010000:00010000:3.0:1789760750.218875:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218888:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218898:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218906:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760750.218911:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218921:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218940:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.218948:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8395 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.219031:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507265664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47422us (47780us total) trans 4294972270 rc 0/0 00000100:00100000:3.0:1789760750.219040:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4980 00000100:00100000:3.0:1789760750.219049:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731441431415 time 731486893275 state 0x2 ref 0 00000100:00100000:3.0:1789760750.222106:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.222261:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265792 00000100:00100000:2.0:1789760750.222271:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265792 00000100:00100000:2.0:1789760750.222324:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731441431415 time 731486893275 state 0x0 ref 1 00000100:00100000:2.0:1789760750.222339:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731487153691 time 731486893275 state 0x1 ref 1 00000100:00100000:2.0:1789760750.222370:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731487153691 time 731537673651 state 0x0 ref 1 00000100:00100000:2.0:1789760750.222392:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507265792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.222440:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20279296, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.222579:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.222597:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760750.222601:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.222608:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.222623:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.281166:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972271 is committed 00002000:00010000:2.0:1789760750.281261:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20279296 -> 20283392 00010000:00010000:2.0:1789760750.281278:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281290:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281299:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281306:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.281311:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281319:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281339:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.281346:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b839c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.281458:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507265792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59056us (59348us total) trans 4294972271 rc 0/0 00000100:00100000:2.0:1789760750.281471:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4981 00000100:00100000:2.0:1789760750.281480:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731487153691 time 731537673651 state 0x2 ref 0 00000100:00100000:0.0:1789760750.284822:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760750.285010:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507265920 00000100:00100000:1.0:1789760750.285021:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507265920 00000100:00100000:1.0:1789760750.285073:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731487153691 time 731537673651 state 0x0 ref 1 00000100:00100000:1.0:1789760750.285091:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731537934067 time 731537673651 state 0x1 ref 1 00000100:00100000:1.0:1789760750.285121:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731537934067 time 731600424440 state 0x0 ref 1 00000100:00100000:1.0:1789760750.285134:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507265920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760750.285227:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20283392, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:1.0:1789760750.285236:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.285251:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760750.285256:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.285263:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.285275:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.300884:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760750.300909:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538278144:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760750.300981:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760750.301056:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278144 00000100:00100000:2.0:1789760750.301063:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278144 00000100:00100000:2.0:1789760750.301109:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789760750.301119:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538278144:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760750.301171:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760750.301217:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538278144:12345-0@lo:41:osp-pre-1-0.0 Request processed in 99us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760750.301224:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789760750.301264:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538278144:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760750.323331:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972272 is committed 00002000:00010000:2.0:1789760750.323435:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20283392 -> 20287488 00010000:00010000:2.0:1789760750.323454:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323467:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323476:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323484:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.323490:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323501:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323522:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.323531:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b83a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.323634:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507265920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38494us (38813us total) trans 4294972272 rc 0/0 00000100:00100000:2.0:1789760750.323645:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4982 00000100:00100000:2.0:1789760750.323655:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731537934067 time 731600424440 state 0x2 ref 0 00000100:00100000:3.0:1789760750.328831:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760750.329129:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266048 00000100:00100000:1.0:1789760750.329138:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266048 00000100:00100000:1.0:1789760750.329189:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731537934067 time 731600424440 state 0x0 ref 1 00000100:00100000:1.0:1789760750.329204:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731600684856 time 731600424440 state 0x1 ref 1 00000100:00100000:1.0:1789760750.329238:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731600684856 time 731644541134 state 0x0 ref 1 00000100:00100000:1.0:1789760750.329251:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507266048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760750.329299:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20287488, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:1.0:1789760750.329309:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.329324:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760750.329328:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.329335:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.329348:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760750.352894:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20287488 -> 20291584 00010000:00010000:1.0:1789760750.352912:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352924:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352932:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352938:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760750.352943:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352950:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352967:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.352974:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b83aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.353058:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507266048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23805us (24233us total) trans 4294972273 rc 0/0 00000100:00100000:1.0:1789760750.353068:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4983 00000100:00100000:1.0:1789760750.353078:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731600684856 time 731644541134 state 0x2 ref 0 00000001:00080000:1.0:1789760750.353114:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972273 is committed 00000100:00100000:3.0:1789760750.361201:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.361462:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266176 00000100:00100000:2.0:1789760750.361473:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266176 00000100:00100000:2.0:1789760750.361526:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731600684856 time 731644541134 state 0x0 ref 1 00000100:00100000:2.0:1789760750.361542:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731644801550 time 731644541134 state 0x1 ref 1 00000100:00100000:2.0:1789760750.361575:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731644801550 time 731676877866 state 0x0 ref 1 00000100:00100000:2.0:1789760750.361804:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.361854:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20291584, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.361864:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.361879:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760750.361884:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.361890:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.361904:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.406085:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972274 is committed 00002000:00010000:3.0:1789760750.406190:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20291584 -> 20295680 00010000:00010000:3.0:1789760750.406204:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406216:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406230:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406236:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760750.406241:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406249:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406266:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.406273:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b83b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.407559:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45750us (46357us total) trans 4294972274 rc 0/0 00000100:00100000:3.0:1789760750.407571:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4984 00000100:00100000:3.0:1789760750.407580:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731644801550 time 731676877866 state 0x2 ref 0 00000100:00100000:1.0:1789760750.413507:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.413566:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266304 00000100:00100000:2.0:1789760750.413577:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266304 00000100:00100000:2.0:1789760750.413632:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731644801550 time 731676877866 state 0x0 ref 1 00000100:00100000:2.0:1789760750.413647:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731677138282 time 731676877866 state 0x1 ref 1 00000100:00100000:2.0:1789760750.413678:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731677138282 time 731728981858 state 0x0 ref 1 00000100:00100000:2.0:1789760750.413688:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.413862:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20295680, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.413870:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.413885:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760750.413889:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.413895:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.413909:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.444365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972275 is committed 00002000:00010000:3.0:1789760750.444465:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20295680 -> 20299776 00010000:00010000:3.0:1789760750.444482:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444502:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444511:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444519:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760750.444524:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444534:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444555:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.444564:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b83b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.444658:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30965us (31156us total) trans 4294972275 rc 0/0 00000100:00100000:3.0:1789760750.444669:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4985 00000100:00100000:3.0:1789760750.444678:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731677138282 time 731728981858 state 0x2 ref 0 00000100:00100000:1.0:1789760750.449289:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.450154:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266432 00000100:00100000:2.0:1789760750.450165:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266432 00000100:00100000:2.0:1789760750.450224:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731677138282 time 731728981858 state 0x0 ref 1 00000100:00100000:2.0:1789760750.450241:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731729242274 time 731728981858 state 0x1 ref 1 00000100:00100000:2.0:1789760750.450336:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731729242274 time 731765638208 state 0x0 ref 1 00000100:00100000:2.0:1789760750.450351:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507266432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.450402:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20299776, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.450410:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.450425:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760750.450429:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.450435:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.450448:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.493845:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760750.497296:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972276 is committed 00002000:00010000:1.0:1789760750.497375:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20299776 -> 20303872 00010000:00010000:1.0:1789760750.497390:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497402:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497411:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497417:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760750.497423:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497432:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497525:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.497533:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b83bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.497656:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507266432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47303us (48372us total) trans 4294972276 rc 0/0 00000100:00100000:1.0:1789760750.497667:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4986 00000100:00100000:1.0:1789760750.497677:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731729242274 time 731765638208 state 0x2 ref 0 00000100:00100000:0.0:1789760750.503119:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760750.503199:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266560 00000100:00100000:3.0:1789760750.503209:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266560 00000100:00100000:3.0:1789760750.503263:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731729242274 time 731765638208 state 0x0 ref 1 00000100:00100000:3.0:1789760750.503277:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731765898624 time 731765638208 state 0x1 ref 1 00000100:00100000:3.0:1789760750.503307:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731765898624 time 731818611004 state 0x0 ref 1 00000100:00100000:3.0:1789760750.503321:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760750.503433:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20303872, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:3.0:1789760750.503443:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.503460:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760750.503465:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.503471:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.503485:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.573382:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972277 is committed 00002000:00010000:2.0:1789760750.574062:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20303872 -> 20307968 00010000:00010000:2.0:1789760750.574531:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574546:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574555:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574563:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.574568:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574579:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574599:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.574607:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b83c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.575129:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71804us (72010us total) trans 4294972277 rc 0/0 00000100:00100000:2.0:1789760750.575140:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4987 00000100:00100000:2.0:1789760750.575160:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731765898624 time 731818611004 state 0x2 ref 0 00000100:00100000:0.0:1789760750.580918:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760750.582433:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266688 00000100:00100000:3.0:1789760750.582448:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266688 00000100:00100000:3.0:1789760750.582510:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731765898624 time 731818611004 state 0x0 ref 1 00000100:00100000:3.0:1789760750.582526:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731818871420 time 731818611004 state 0x1 ref 1 00000100:00100000:3.0:1789760750.582562:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731818871420 time 731897864618 state 0x0 ref 1 00000100:00100000:3.0:1789760750.582580:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760750.582633:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20307968, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:3.0:1789760750.582668:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.582684:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760750.582689:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.582871:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.582890:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.641026:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972278 is committed 00002000:00010000:2.0:1789760750.641223:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20307968 -> 20312064 00010000:00010000:2.0:1789760750.641242:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641258:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641268:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641276:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.641282:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641293:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641316:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.641324:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b83cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.641446:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507266688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58865us (60529us total) trans 4294972278 rc 0/0 00000100:00100000:2.0:1789760750.641458:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4988 00000100:00100000:2.0:1789760750.641471:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731818871420 time 731897864618 state 0x2 ref 0 00000100:00100000:3.0:1789760750.646526:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760750.646648:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266816 00000100:00100000:1.0:1789760750.646662:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266816 00000100:00100000:1.0:1789760750.646766:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731818871420 time 731897864618 state 0x0 ref 1 00000100:00100000:1.0:1789760750.646786:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731898125034 time 731897864618 state 0x1 ref 1 00000100:00100000:1.0:1789760750.646927:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731898125034 time 731962229090 state 0x0 ref 1 00000100:00100000:1.0:1789760750.646943:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507266816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760750.647129:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20312064, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:1.0:1789760750.647140:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.647160:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760750.647165:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.647173:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.647191:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.718249:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20312064 -> 20316160 00010000:00010000:3.0:1789760750.718278:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718295:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718305:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718320:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760750.718326:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718336:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718363:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.718370:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b83d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.718504:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507266816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71556us (71985us total) trans 4294972279 rc 0/0 00000100:00100000:3.0:1789760750.718516:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4989 00000100:00100000:3.0:1789760750.718527:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731898125034 time 731962229090 state 0x2 ref 0 00000001:00080000:1.0:1789760750.719381:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972279 is committed 00000100:00100000:3.0:1789760750.721403:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.722063:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507266944 00000100:00100000:2.0:1789760750.722078:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507266944 00000100:00100000:2.0:1789760750.722158:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731898125034 time 731962229090 state 0x0 ref 1 00000100:00100000:2.0:1789760750.722178:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731962489506 time 731962229090 state 0x1 ref 1 00000100:00100000:2.0:1789760750.722256:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731962489506 time 732037558812 state 0x0 ref 1 00000100:00100000:2.0:1789760750.722272:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507266944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.722342:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20316160, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.722353:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.722374:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760750.722380:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.722388:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.722406:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.761256:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972280 is committed 00002000:00010000:3.0:1789760750.761364:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20316160 -> 20320256 00010000:00010000:3.0:1789760750.761384:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761399:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761415:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761423:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760750.761429:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761438:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761463:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761472:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b83db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.761629:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507266944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39353us (40235us total) trans 4294972280 rc 0/0 00000100:00100000:3.0:1789760750.761642:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4990 00000100:00100000:3.0:1789760750.761654:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731962489506 time 732037558812 state 0x2 ref 0 00000100:00100000:0.0:1789760750.765178:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.765271:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267072 00000100:00100000:2.0:1789760750.765284:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267072 00000100:00100000:2.0:1789760750.765353:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 731962489506 time 732037558812 state 0x0 ref 1 00000100:00100000:2.0:1789760750.765370:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732037819228 time 732037558812 state 0x1 ref 1 00000100:00100000:2.0:1789760750.765405:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732037819228 time 732080707580 state 0x0 ref 1 00000100:00100000:2.0:1789760750.765418:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.765478:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20320256, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.765488:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.765507:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760750.765512:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.765519:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.765535:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.810940:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20320256 -> 20324352 00010000:00010000:2.0:1789760750.810968:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.810982:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.810993:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.811002:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.811009:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.811019:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.811045:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.811053:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.811194:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507267072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45770us (46022us total) trans 4294972281 rc 0/0 00000100:00100000:2.0:1789760750.811207:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4991 00000100:00100000:2.0:1789760750.811220:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732037819228 time 732080707580 state 0x2 ref 0 00000001:00080000:3.0:1789760750.811467:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972281 is committed 00000100:00100000:3.0:1789760750.814369:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760750.815414:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267200 00000100:00100000:0.0:1789760750.815428:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267200 00000100:00100000:0.0:1789760750.815826:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732037819228 time 732080707580 state 0x0 ref 1 00000100:00100000:0.0:1789760750.815849:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732080967996 time 732080707580 state 0x1 ref 1 00000100:00100000:0.0:1789760750.815880:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732080967996 time 732131183224 state 0x0 ref 1 00000100:00100000:0.0:1789760750.815897:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760750.815967:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20324352, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:0.0:1789760750.815979:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.816000:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760750.816005:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.816014:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.816033:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.818132:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538278272:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760750.818235:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760750.818379:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278272 00000100:00100000:0.0:1789760750.818391:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278272 00000100:00100000:0.0:1789760750.818456:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789760750.818468:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538278272:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760750.818910:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760750.819027:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234674a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538278272:12345-0@lo:41:osp-pre-0-1.0 Request processed in 561us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760750.819036:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:2.0:1789760750.822338:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538278272:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760750.886509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972282 is committed 00002000:00010000:0.0:1789760750.886687:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20324352 -> 20328448 00010000:00010000:0.0:1789760750.886746:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886888:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886899:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886907:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760750.886912:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886922:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886950:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.886959:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b83e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.887085:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71186us (72726us total) trans 4294972282 rc 0/0 00000100:00100000:0.0:1789760750.887096:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4992 00000100:00100000:0.0:1789760750.887107:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732080967996 time 732131183224 state 0x2 ref 0 00000100:00100000:0.0:1789760750.891183:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760750.891554:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267328 00000100:00100000:3.0:1789760750.891568:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267328 00000100:00100000:3.0:1789760750.891643:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732080967996 time 732131183224 state 0x0 ref 1 00000100:00100000:3.0:1789760750.891664:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732131443640 time 732131183224 state 0x1 ref 1 00000100:00100000:3.0:1789760750.891734:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732131443640 time 732207036173 state 0x0 ref 1 00000100:00100000:3.0:1789760750.891749:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507267328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760750.891820:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20328448, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:3.0:1789760750.891832:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.891855:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760750.891861:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.891870:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.891888:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.941363:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972283 is committed 00002000:00010000:1.0:1789760750.941626:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20328448 -> 20332544 00010000:00010000:1.0:1789760750.941646:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941845:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941856:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941865:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760750.941870:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941881:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941908:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.941917:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b83f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.942050:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507267328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50296us (50874us total) trans 4294972283 rc 0/0 00000100:00100000:1.0:1789760750.942063:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4993 00000100:00100000:1.0:1789760750.942075:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732131443640 time 732207036173 state 0x2 ref 0 00000100:00100000:1.0:1789760750.944793:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760750.944901:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267456 00000100:00100000:2.0:1789760750.944921:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267456 00000100:00100000:2.0:1789760750.945000:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732131443640 time 732207036173 state 0x0 ref 1 00000100:00100000:2.0:1789760750.945022:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732207296589 time 732207036173 state 0x1 ref 1 00000100:00100000:2.0:1789760750.945058:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732207296589 time 732260360752 state 0x0 ref 1 00000100:00100000:2.0:1789760750.945072:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760750.945141:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20332544, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:2.0:1789760750.945153:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.945173:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760750.945178:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.945186:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.945202:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.986395:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20332544 -> 20336640 00010000:00010000:2.0:1789760750.986423:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986439:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986449:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986457:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760750.986463:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986473:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986497:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.986506:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b83f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.986621:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507267456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41544us (41836us total) trans 4294972284 rc 0/0 00000100:00100000:2.0:1789760750.986633:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4994 00000100:00100000:2.0:1789760750.986645:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732207296589 time 732260360752 state 0x2 ref 0 00000001:00080000:3.0:1789760750.986880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972284 is committed 00000100:00100000:3.0:1789760750.989071:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760750.989565:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267584 00000100:00100000:0.0:1789760750.989579:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267584 00000100:00100000:0.0:1789760750.994489:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732207296589 time 732260360752 state 0x0 ref 1 00000100:00100000:0.0:1789760750.994525:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732260621168 time 732260360752 state 0x1 ref 1 00000100:00100000:1.0:1789760750.994747:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732260621168 time 732310045920 state 0x0 ref 1 00000100:00100000:1.0:1789760750.994790:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507267584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760750.994862:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20336640, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa007 00010000:00010000:1.0:1789760750.994875:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.994896:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760750.994901:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.994911:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.994928:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.059823:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972285 is committed 00002000:00010000:2.0:1789760751.060291:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20336640 -> 20340736 00002000:00010000:2.0:1789760751.060297:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:2.0:1789760751.060299:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:2.0:1789760751.060318:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060333:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060343:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060352:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.060358:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060370:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060395:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.060405:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b83fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.060530:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507267584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65761us (71464us total) trans 4294972285 rc 0/0 00000100:00100000:2.0:1789760751.060543:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4995 00000100:00100000:2.0:1789760751.060555:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732260621168 time 732310045920 state 0x2 ref 0 00000100:00100000:0.0:1789760751.069765:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760751.069971:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267712 00000100:00100000:1.0:1789760751.069987:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267712 00000100:00100000:1.0:1789760751.070082:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732260621168 time 732310045920 state 0x0 ref 1 00000100:00100000:1.0:1789760751.070101:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732310306336 time 732310045920 state 0x1 ref 1 00000100:00080000:0.0:1789760751.070195:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760751.070840:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732310306336 time 732386139052 state 0x0 ref 1 00000100:00100000:1.0:1789760751.070864:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760751.070948:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20340736, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760751.070960:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.070980:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760751.070986:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.070994:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.071024:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.118056:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972286 is committed 00002000:00010000:2.0:1789760751.118257:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20340736 -> 20344832 00010000:00010000:2.0:1789760751.118275:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118295:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118309:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118317:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.118322:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118332:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118352:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.118370:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8405 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.118469:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507267712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47608us (48774us total) trans 4294972286 rc 0/0 00000100:00100000:2.0:1789760751.118480:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4996 00000100:00100000:2.0:1789760751.118490:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732310306336 time 732386139052 state 0x2 ref 0 00000100:00100000:1.0:1789760751.123855:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.123923:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267840 00000100:00100000:2.0:1789760751.123939:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267840 00000100:00100000:2.0:1789760751.124004:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732310306336 time 732386139052 state 0x0 ref 1 00000100:00100000:2.0:1789760751.124020:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732386399468 time 732386139052 state 0x1 ref 1 00000100:00100000:2.0:1789760751.124057:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732386399468 time 732439360212 state 0x0 ref 1 00000100:00100000:2.0:1789760751.124070:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507267840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.124117:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20344832, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.124126:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.124142:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760751.124146:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.124153:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.124167:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.202135:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972287 is committed 00002000:00010000:2.0:1789760751.203397:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20344832 -> 20348928 00010000:00010000:2.0:1789760751.203422:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203438:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203446:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203454:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.203460:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203469:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203492:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.203499:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b840c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.203679:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507267840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79603us (79831us total) trans 4294972287 rc 0/0 00000100:00100000:2.0:1789760751.203743:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4997 00000100:00100000:2.0:1789760751.203756:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732386399468 time 732439360212 state 0x2 ref 0 00000100:00100000:1.0:1789760751.213917:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.214043:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507267968 00000100:00100000:2.0:1789760751.214057:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507267968 00000100:00100000:2.0:1789760751.214130:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732386399468 time 732439360212 state 0x0 ref 1 00000100:00100000:2.0:1789760751.214148:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732439620628 time 732439360212 state 0x1 ref 1 00000100:00100000:2.0:1789760751.214191:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732439620628 time 732529493522 state 0x0 ref 1 00000100:00100000:2.0:1789760751.214206:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507267968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.214272:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20348928, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.214284:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.214304:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760751.214310:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.214318:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.214334:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.260148:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20348928 -> 20353024 00010000:00010000:1.0:1789760751.260172:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260187:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260262:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260273:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.260279:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260291:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260317:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.260326:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8413 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.260456:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507267968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46246us (46547us total) trans 4294972288 rc 0/0 00000100:00100000:1.0:1789760751.260468:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4998 00000100:00100000:1.0:1789760751.260480:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732439620628 time 732529493522 state 0x2 ref 0 00000001:00080000:1.0:1789760751.260532:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972288 is committed 00000100:00100000:1.0:1789760751.264490:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.264596:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268096 00000100:00100000:2.0:1789760751.264609:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268096 00000100:00100000:2.0:1789760751.264679:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732439620628 time 732529493522 state 0x0 ref 1 00000100:00100000:2.0:1789760751.264756:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732529753938 time 732529493522 state 0x1 ref 1 00000100:00100000:2.0:1789760751.264848:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732529753938 time 732580149240 state 0x0 ref 1 00000100:00100000:2.0:1789760751.264865:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507268096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.264931:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20353024, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.264942:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.264962:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760751.264967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.264975:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.264994:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760751.322338:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20353024 -> 20357120 00010000:00010000:2.0:1789760751.322364:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322381:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322391:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322400:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.322405:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322415:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322437:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.322445:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b841a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.322572:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507268096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57704us (58091us total) trans 4294972289 rc 0/0 00000100:00100000:2.0:1789760751.322584:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 4999 00000100:00100000:2.0:1789760751.322596:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732529753938 time 732580149240 state 0x2 ref 0 00000001:00080000:0.0:1789760751.323060:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00000100:00100000:2.0:1789760751.328561:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760751.328873:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268224 00000100:00100000:0.0:1789760751.328889:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268224 00000100:00100000:0.0:1789760751.329215:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732529753938 time 732580149240 state 0x0 ref 1 00000100:00100000:0.0:1789760751.329238:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732580409656 time 732580149240 state 0x1 ref 1 00000100:00100000:1.0:1789760751.329621:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732580409656 time 732644919548 state 0x0 ref 1 00000100:00100000:1.0:1789760751.329656:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507268224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760751.329983:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20357120, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760751.329995:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.330016:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760751.330022:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.330030:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.330048:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.383913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00002000:00010000:1.0:1789760751.384248:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20357120 -> 20361216 00010000:00010000:1.0:1789760751.384270:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384287:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384297:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384305:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.384311:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384321:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384346:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.384354:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8421 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.384523:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507268224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54868us (55973us total) trans 4294972290 rc 0/0 00000100:00100000:1.0:1789760751.384536:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5000 00000100:00100000:1.0:1789760751.384548:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732580409656 time 732644919548 state 0x2 ref 0 00000100:00100000:1.0:1789760751.398958:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760751.399134:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268352 00000100:00100000:3.0:1789760751.399149:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268352 00000100:00100000:3.0:1789760751.403162:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732580409656 time 732644919548 state 0x0 ref 1 00000100:00100000:3.0:1789760751.403194:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732645179964 time 732644919548 state 0x1 ref 1 00000100:00100000:3.0:1789760751.403240:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732645179964 time 732718543531 state 0x0 ref 1 00000100:00100000:3.0:1789760751.403269:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760751.403341:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20361216, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:3.0:1789760751.403352:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.403372:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760751.403377:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.403384:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.403400:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.441758:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972291 is committed 00002000:00010000:3.0:1789760751.441991:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20361216 -> 20365312 00010000:00010000:3.0:1789760751.442012:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442027:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442038:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442047:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760751.442053:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442064:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442164:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.442174:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8428 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.442340:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39081us (43388us total) trans 4294972291 rc 0/0 00000100:00100000:3.0:1789760751.442354:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5001 00000100:00100000:3.0:1789760751.442366:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732645179964 time 732718543531 state 0x2 ref 0 00000100:00100000:3.0:1789760751.448193:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.449158:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268480 00000100:00100000:2.0:1789760751.449172:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268480 00000100:00100000:2.0:1789760751.449247:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732645179964 time 732718543531 state 0x0 ref 1 00000100:00100000:2.0:1789760751.449266:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732718803947 time 732718543531 state 0x1 ref 1 00000100:00100000:2.0:1789760751.449335:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732718803947 time 732764637857 state 0x0 ref 1 00000100:00100000:2.0:1789760751.449350:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.449411:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20365312, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.449423:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.449442:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760751.449448:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.449456:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.449478:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.507332:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972292 is committed 00002000:00010000:1.0:1789760751.507510:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20365312 -> 20369408 00010000:00010000:1.0:1789760751.507528:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507544:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507555:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507562:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.507567:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507577:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507600:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.507607:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b842f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.507960:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58604us (59772us total) trans 4294972292 rc 0/0 00000100:00100000:1.0:1789760751.507973:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5002 00000100:00100000:1.0:1789760751.507987:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732718803947 time 732764637857 state 0x2 ref 0 00000100:00100000:1.0:1789760751.511216:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.511329:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268608 00000100:00100000:2.0:1789760751.511342:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268608 00000100:00100000:2.0:1789760751.511411:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732718803947 time 732764637857 state 0x0 ref 1 00000100:00100000:2.0:1789760751.511429:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732764898273 time 732764637857 state 0x1 ref 1 00000100:00100000:2.0:1789760751.511505:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732764898273 time 732826807685 state 0x0 ref 1 00000100:00100000:2.0:1789760751.511519:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.511581:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20369408, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.511591:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.511610:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760751.511616:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.511623:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.511639:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.526179:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760751.568086:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972293 is committed 00002000:00010000:1.0:1789760751.568235:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20369408 -> 20373504 00010000:00010000:1.0:1789760751.568255:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568271:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568282:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568290:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.568296:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568306:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568331:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.568340:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8436 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.568576:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57052us (57365us total) trans 4294972293 rc 0/0 00000100:00100000:1.0:1789760751.568590:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5003 00000100:00100000:1.0:1789760751.568601:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732764898273 time 732826807685 state 0x2 ref 0 00000100:00100000:2.0:1789760751.572166:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760751.572240:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268736 00000100:00100000:1.0:1789760751.572252:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268736 00000100:00100000:1.0:1789760751.572315:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732764898273 time 732826807685 state 0x0 ref 1 00000100:00100000:1.0:1789760751.572332:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732827068101 time 732826807685 state 0x1 ref 1 00000100:00100000:1.0:1789760751.572362:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732827068101 time 732887665768 state 0x0 ref 1 00000100:00100000:1.0:1789760751.572375:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760751.572436:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20373504, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760751.572447:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.572466:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760751.572471:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.572478:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.572496:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.663137:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972294 is committed 00002000:00010000:2.0:1789760751.663486:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20373504 -> 20377600 00010000:00010000:2.0:1789760751.663505:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663521:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663531:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663540:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.663545:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663554:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663578:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.663586:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b843d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.663768:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91385us (91608us total) trans 4294972294 rc 0/0 00000100:00100000:2.0:1789760751.663780:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5004 00000100:00100000:2.0:1789760751.663792:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732827068101 time 732887665768 state 0x2 ref 0 00000100:00100000:2.0:1789760751.666977:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760751.667442:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268864 00000100:00100000:1.0:1789760751.667457:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268864 00000100:00100000:1.0:1789760751.667535:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732827068101 time 732887665768 state 0x0 ref 1 00000100:00100000:1.0:1789760751.667555:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732887926184 time 732887665768 state 0x1 ref 1 00000100:00100000:1.0:1789760751.667587:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732887926184 time 732982890175 state 0x0 ref 1 00000100:00100000:1.0:1789760751.667601:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507268864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760751.667671:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20377600, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760751.667681:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.667755:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760751.667762:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.667770:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.667789:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.710301:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972295 is committed 00002000:00010000:1.0:1789760751.710462:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20377600 -> 20381696 00010000:00010000:1.0:1789760751.710483:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710498:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710515:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710523:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.710529:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710539:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710566:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.710574:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8444 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.710742:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507268864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43089us (43727us total) trans 4294972295 rc 0/0 00000100:00100000:1.0:1789760751.710760:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5005 00000100:00100000:1.0:1789760751.710772:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732887926184 time 732982890175 state 0x2 ref 0 00000100:00100000:1.0:1789760751.713674:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.715042:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507268992 00000100:00100000:2.0:1789760751.715059:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507268992 00000100:00100000:0.0:1789760751.715233:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732887926184 time 732982890175 state 0x0 ref 1 00000100:00100000:0.0:1789760751.715253:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732983150591 time 732982890175 state 0x1 ref 1 00000100:00100000:2.0:1789760751.715370:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732983150591 time 733030669238 state 0x0 ref 1 00000100:00100000:2.0:1789760751.715390:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507268992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.715456:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20381696, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.715467:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.715486:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760751.715491:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.715498:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.715516:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.747438:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20381696 -> 20385792 00010000:00010000:3.0:1789760751.747482:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747496:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747504:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747512:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760751.747517:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747527:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747550:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.747557:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b844b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.747685:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507268992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32294us (34017us total) trans 4294972296 rc 0/0 00000100:00100000:3.0:1789760751.747741:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5006 00000100:00100000:3.0:1789760751.748257:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732983150591 time 733030669238 state 0x2 ref 0 00000001:00080000:1.0:1789760751.748371:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972296 is committed 00000100:00100000:2.0:1789760751.751005:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760751.751420:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269120 00000100:00100000:1.0:1789760751.751433:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269120 00000100:00100000:1.0:1789760751.751580:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 732983150591 time 733030669238 state 0x0 ref 1 00000100:00100000:1.0:1789760751.751599:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733030929654 time 733030669238 state 0x1 ref 1 00000100:00100000:1.0:1789760751.751685:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733030929654 time 733066986791 state 0x0 ref 1 00000100:00100000:1.0:1789760751.751740:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507269120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760751.751808:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20385792, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760751.751819:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.751838:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760751.751844:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.751852:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.751869:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.793227:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972297 is committed 00002000:00010000:1.0:1789760751.794300:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20385792 -> 20389888 00010000:00010000:1.0:1789760751.794327:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794344:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794355:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794366:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.794372:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794384:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794408:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.794417:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8452 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.794666:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507269120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42921us (43667us total) trans 4294972297 rc 0/0 00000100:00100000:1.0:1789760751.794678:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5007 00000100:00100000:1.0:1789760751.794737:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733030929654 time 733066986791 state 0x2 ref 0 00000100:00100000:1.0:1789760751.801518:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.801732:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269248 00000100:00100000:2.0:1789760751.801747:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269248 00000100:00100000:2.0:1789760751.801823:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733030929654 time 733066986791 state 0x0 ref 1 00000100:00100000:2.0:1789760751.801843:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733067247207 time 733066986791 state 0x1 ref 1 00000100:00100000:2.0:1789760751.801874:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733067247207 time 733117177182 state 0x0 ref 1 00000100:00100000:2.0:1789760751.801887:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507269248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.801957:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20389888, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.801967:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.801986:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760751.801992:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.802000:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.802016:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.833248:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972298 is committed 00002000:00010000:2.0:1789760751.833397:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20389888 -> 20393984 00010000:00010000:2.0:1789760751.833417:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833433:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833688:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833732:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760751.833739:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833823:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833851:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.833859:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8459 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.834045:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507269248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32152us (32536us total) trans 4294972298 rc 0/0 00000100:00100000:2.0:1789760751.834059:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5008 00000100:00100000:2.0:1789760751.834071:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733067247207 time 733117177182 state 0x2 ref 0 00000100:00100000:0.0:1789760751.837671:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760751.837775:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269376 00000100:00100000:3.0:1789760751.837788:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269376 00000100:00100000:3.0:1789760751.837860:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733067247207 time 733117177182 state 0x0 ref 1 00000100:00100000:3.0:1789760751.837877:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733117437598 time 733117177182 state 0x1 ref 1 00000100:00100000:3.0:1789760751.837915:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733117437598 time 733153218387 state 0x0 ref 1 00000100:00100000:3.0:1789760751.837934:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507269376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760751.838008:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20393984, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:3.0:1789760751.838018:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.838036:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760751.838041:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.838049:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.838065:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.942418:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972299 is committed 00002000:00010000:1.0:1789760751.942818:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20393984 -> 20398080 00010000:00010000:1.0:1789760751.942841:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942855:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942872:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942880:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760751.942886:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942895:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942916:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.942924:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8460 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.943043:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507269376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 105107us (105382us total) trans 4294972299 rc 0/0 00000100:00100000:1.0:1789760751.943054:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5009 00000100:00100000:1.0:1789760751.943064:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733117437598 time 733153218387 state 0x2 ref 0 00000100:00100000:1.0:1789760751.971255:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760751.971363:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269504 00000100:00100000:2.0:1789760751.971378:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269504 00000100:00100000:2.0:1789760751.971456:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733117437598 time 733153218387 state 0x0 ref 1 00000100:00100000:2.0:1789760751.971478:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733153478803 time 733153218387 state 0x1 ref 1 00000100:00100000:2.0:1789760751.971521:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733153478803 time 733286823092 state 0x0 ref 1 00000100:00100000:2.0:1789760751.971536:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507269504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760751.971611:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20398080, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:2.0:1789760751.971623:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.971642:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760751.971648:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.971656:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.971673:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.036215:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972300 is committed 00002000:00010000:3.0:1789760752.036303:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20398080 -> 20402176 00010000:00010000:3.0:1789760752.036323:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036338:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036348:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036355:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.036361:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036373:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036397:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.036405:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8467 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.036527:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507269504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64986us (65279us total) trans 4294972300 rc 0/0 00000100:00100000:3.0:1789760752.036541:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5010 00000100:00100000:3.0:1789760752.036553:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733153478803 time 733286823092 state 0x2 ref 0 00000100:00100000:2.0:1789760752.049230:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.049435:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269632 00000100:00100000:1.0:1789760752.049449:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269632 00000100:00100000:1.0:1789760752.049528:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733153478803 time 733286823092 state 0x0 ref 1 00000100:00100000:1.0:1789760752.049552:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733287083508 time 733286823092 state 0x1 ref 1 00000100:00100000:1.0:1789760752.049630:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733287083508 time 733364932797 state 0x0 ref 1 00000100:00100000:1.0:1789760752.049646:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507269632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.049756:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20402176, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa007 00010000:00010000:1.0:1789760752.049767:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.049788:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.049793:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.049802:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.049820:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.121234:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00002000:00010000:1.0:1789760752.121470:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20402176 -> 20406272 00002000:00010000:1.0:1789760752.121476:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:1.0:1789760752.121479:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:1.0:1789760752.121497:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121513:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121530:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121539:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.121545:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121556:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121582:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.121591:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b846e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.122172:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507269632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72521us (72952us total) trans 4294972301 rc 0/0 00000100:00100000:1.0:1789760752.122186:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5011 00000100:00100000:1.0:1789760752.122198:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733287083508 time 733364932797 state 0x2 ref 0 00000100:00100000:2.0:1789760752.128882:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.128999:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269760 00000100:00100000:1.0:1789760752.129014:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269760 00000100:00100000:1.0:1789760752.129089:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733287083508 time 733364932797 state 0x0 ref 1 00000100:00100000:1.0:1789760752.129108:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733365193213 time 733364932797 state 0x1 ref 1 00000100:00100000:1.0:1789760752.129147:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733365193213 time 733444449033 state 0x0 ref 1 00000100:00100000:1.0:1789760752.129161:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507269760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.129231:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20406272, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.129242:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.129262:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.129268:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.129275:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.129293:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.225286:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20406272 -> 20410368 00010000:00010000:2.0:1789760752.225313:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225327:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225336:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225344:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760752.225350:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225362:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225387:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.225395:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8475 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.225515:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507269760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96350us (96644us total) trans 4294972302 rc 0/0 00000100:00100000:2.0:1789760752.225527:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5012 00000100:00100000:2.0:1789760752.225539:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733365193213 time 733444449033 state 0x2 ref 0 00000001:00080000:3.0:1789760752.228846:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00000100:00100000:2.0:1789760752.229288:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760752.229548:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507269888 00000100:00100000:3.0:1789760752.229560:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507269888 00000100:00100000:3.0:1789760752.229632:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733365193213 time 733444449033 state 0x0 ref 1 00000100:00100000:3.0:1789760752.229651:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733444709449 time 733444449033 state 0x1 ref 1 00000100:00100000:3.0:1789760752.229946:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733444709449 time 733545246650 state 0x0 ref 1 00000100:00100000:3.0:1789760752.229962:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507269888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760752.230030:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20410368, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:3.0:1789760752.230040:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.230059:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760752.230065:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.230072:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.230088:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.269095:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972303 is committed 00002000:00010000:0.0:1789760752.269288:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20410368 -> 20414464 00010000:00010000:0.0:1789760752.269308:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269324:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269335:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269344:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760752.269349:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269360:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269385:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.269394:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b847c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.269562:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507269888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39595us (40274us total) trans 4294972303 rc 0/0 00000100:00100000:0.0:1789760752.269575:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5013 00000100:00100000:0.0:1789760752.269587:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733444709449 time 733545246650 state 0x2 ref 0 00000100:00100000:2.0:1789760752.273290:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.273436:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270016 00000100:00100000:1.0:1789760752.273449:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270016 00000100:00100000:1.0:1789760752.273517:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733444709449 time 733545246650 state 0x0 ref 1 00000100:00100000:1.0:1789760752.273535:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733545507066 time 733545246650 state 0x1 ref 1 00000100:00100000:1.0:1789760752.273614:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733545507066 time 733588915848 state 0x0 ref 1 00000100:00100000:1.0:1789760752.273629:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.273739:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20414464, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.273758:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.273778:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.273783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.273791:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.273809:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.321835:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00002000:00010000:1.0:1789760752.321992:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20414464 -> 20418560 00010000:00010000:1.0:1789760752.322012:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322028:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322038:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322047:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.322052:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322063:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322089:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.322098:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8483 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.322243:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48609us (48959us total) trans 4294972304 rc 0/0 00000100:00100000:1.0:1789760752.322257:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5014 00000100:00100000:1.0:1789760752.322270:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733545507066 time 733588915848 state 0x2 ref 0 00000100:00100000:3.0:1789760752.325398:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760752.326015:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270144 00000100:00100000:0.0:1789760752.326031:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270144 00000100:00100000:1.0:1789760752.326397:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733545507066 time 733588915848 state 0x0 ref 1 00000100:00100000:1.0:1789760752.326492:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733589176264 time 733588915848 state 0x1 ref 1 00000100:00100000:1.0:1789760752.326575:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733589176264 time 733641876371 state 0x0 ref 1 00000100:00100000:1.0:1789760752.326591:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.326660:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20418560, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.326671:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.326736:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.326751:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.326761:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.326779:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.377949:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972305 is committed 00002000:00010000:1.0:1789760752.378099:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20418560 -> 20422656 00010000:00010000:1.0:1789760752.378122:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378140:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378162:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378172:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.378179:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378191:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378217:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.378226:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b848a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.378360:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51763us (52970us total) trans 4294972305 rc 0/0 00000100:00100000:1.0:1789760752.378373:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5015 00000100:00100000:1.0:1789760752.378386:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733589176264 time 733641876371 state 0x2 ref 0 00000100:00100000:3.0:1789760752.381617:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.381862:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270272 00000100:00100000:1.0:1789760752.381875:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270272 00000100:00100000:1.0:1789760752.381951:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733589176264 time 733641876371 state 0x0 ref 1 00000100:00100000:1.0:1789760752.381971:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733642136787 time 733641876371 state 0x1 ref 1 00000100:00100000:1.0:1789760752.381998:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733642136787 time 733697301239 state 0x0 ref 1 00000100:00100000:1.0:1789760752.382011:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507270272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.382081:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20422656, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.382092:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.382113:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.382118:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.382126:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.382143:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.426535:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972306 is committed 00002000:00010000:1.0:1789760752.426774:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20422656 -> 20426752 00010000:00010000:1.0:1789760752.426793:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426809:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426819:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426827:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.426833:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426844:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426867:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.426875:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8491 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.426990:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507270272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44973us (45378us total) trans 4294972306 rc 0/0 00000100:00100000:1.0:1789760752.427006:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5016 00000100:00100000:1.0:1789760752.427019:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733642136787 time 733697301239 state 0x2 ref 0 00000100:00100000:2.0:1789760752.429970:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.430230:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270400 00000100:00100000:1.0:1789760752.430243:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270400 00000100:00100000:1.0:1789760752.430313:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733642136787 time 733697301239 state 0x0 ref 1 00000100:00100000:1.0:1789760752.430330:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733697561655 time 733697301239 state 0x1 ref 1 00000100:00100000:1.0:1789760752.430359:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733697561655 time 733745663116 state 0x0 ref 1 00000100:00100000:1.0:1789760752.430372:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.430443:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20426752, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.430454:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.430472:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.430478:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.430486:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.430504:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.467620:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972307 is committed 00002000:00010000:1.0:1789760752.467795:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:1.0:1789760752.467814:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467831:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467841:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467849:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.467854:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467865:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467890:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.467899:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8498 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.468023:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37644us (38060us total) trans 4294972307 rc 0/0 00000100:00100000:1.0:1789760752.468035:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5017 00000100:00100000:1.0:1789760752.468047:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733697561655 time 733745663116 state 0x2 ref 0 00000100:00100000:3.0:1789760752.471845:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.471971:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270528 00000100:00100000:1.0:1789760752.471985:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270528 00000100:00100000:1.0:1789760752.472059:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733697561655 time 733745663116 state 0x0 ref 1 00000100:00100000:1.0:1789760752.472078:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733745923532 time 733745663116 state 0x1 ref 1 00000100:00100000:1.0:1789760752.472106:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733745923532 time 733787408944 state 0x0 ref 1 00000100:00100000:1.0:1789760752.472119:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507270528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.472357:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20430848, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.472370:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.472390:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.472395:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.472403:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.472418:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.510363:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20430848 -> 20434944 00010000:00010000:2.0:1789760752.510392:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510408:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510420:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510430:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760752.510447:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510458:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510485:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.510493:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b849f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.510625:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507270528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38500us (38788us total) trans 4294972308 rc 0/0 00000100:00100000:2.0:1789760752.510637:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5018 00000100:00100000:2.0:1789760752.510651:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733745923532 time 733787408944 state 0x2 ref 0 00000001:00080000:3.0:1789760752.513341:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972308 is committed 00000100:00100000:2.0:1789760752.513930:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.514024:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270656 00000100:00100000:1.0:1789760752.514037:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270656 00000100:00100000:1.0:1789760752.514112:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733745923532 time 733787408944 state 0x0 ref 1 00000100:00100000:1.0:1789760752.514131:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733787669360 time 733787408944 state 0x1 ref 1 00000100:00100000:1.0:1789760752.514256:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733787669360 time 733829557731 state 0x0 ref 1 00000100:00100000:1.0:1789760752.514275:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507270656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.514340:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20434944, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.514351:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.514373:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.514379:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.514386:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.514403:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.542101:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760752.547905:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20434944 -> 20439040 00010000:00010000:3.0:1789760752.547935:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.547951:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.547962:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.547974:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.547980:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.547989:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.548014:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.548022:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b84a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.548225:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507270656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33873us (34222us total) trans 4294972309 rc 0/0 00000100:00100000:3.0:1789760752.548240:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5019 00000100:00100000:3.0:1789760752.548254:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733787669360 time 733829557731 state 0x2 ref 0 00000100:00100000:3.0:1789760752.551451:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.551920:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270784 00000100:00100000:1.0:1789760752.551935:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270784 00000100:00100000:1.0:1789760752.552014:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733787669360 time 733829557731 state 0x0 ref 1 00000100:00100000:1.0:1789760752.552033:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733829818147 time 733829557731 state 0x1 ref 1 00000100:00100000:1.0:1789760752.552075:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733829818147 time 733867377905 state 0x0 ref 1 00000100:00100000:1.0:1789760752.552090:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507270784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.552167:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20439040, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.552178:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.552203:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760752.552208:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.552215:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.552233:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.555111:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972309 is committed 00002000:00010000:1.0:1789760752.618602:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20439040 -> 20443136 00010000:00010000:1.0:1789760752.618628:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618643:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618653:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618660:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.618666:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618677:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618732:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.618750:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.618863:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507270784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66768us (67420us total) trans 4294972310 rc 0/0 00000100:00100000:1.0:1789760752.618874:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5020 00000100:00100000:1.0:1789760752.618886:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733829818147 time 733867377905 state 0x2 ref 0 00000001:00080000:0.0:1789760752.619077:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972310 is committed 00000100:00100000:1.0:1789760752.622028:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760752.622184:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507270912 00000100:00100000:3.0:1789760752.622197:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507270912 00000100:00100000:3.0:1789760752.622274:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733829818147 time 733867377905 state 0x0 ref 1 00000100:00100000:3.0:1789760752.622294:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733867638321 time 733867377905 state 0x1 ref 1 00000100:00100000:3.0:1789760752.622335:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733867638321 time 733937637301 state 0x0 ref 1 00000100:00100000:3.0:1789760752.622352:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507270912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760752.622485:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20443136, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:3.0:1789760752.622500:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.622517:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760752.622521:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.622527:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.622540:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.678656:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972311 is committed 00002000:00010000:2.0:1789760752.678942:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20443136 -> 20447232 00010000:00010000:2.0:1789760752.678965:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.678980:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.678996:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.679004:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760752.679010:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.679020:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.679043:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.679050:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b84b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.679169:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507270912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56813us (57147us total) trans 4294972311 rc 0/0 00000100:00100000:2.0:1789760752.679180:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5021 00000100:00100000:2.0:1789760752.679192:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733867638321 time 733937637301 state 0x2 ref 0 00000100:00100000:3.0:1789760752.685303:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.685606:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271040 00000100:00100000:1.0:1789760752.685619:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271040 00000100:00100000:1.0:1789760752.685687:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733867638321 time 733937637301 state 0x0 ref 1 00000100:00100000:1.0:1789760752.685751:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733937897717 time 733937637301 state 0x1 ref 1 00000100:00100000:1.0:1789760752.685887:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733937897717 time 734001189617 state 0x0 ref 1 00000100:00100000:1.0:1789760752.685902:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.685972:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20447232, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.685984:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686005:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760752.686011:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.686020:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686039:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.737183:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972312 is committed 00002000:00010000:3.0:1789760752.737446:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20447232 -> 20451328 00010000:00010000:3.0:1789760752.737467:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737482:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737493:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737501:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.737507:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737521:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737546:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.737554:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b84bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.738355:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52444us (53057us total) trans 4294972312 rc 0/0 00000100:00100000:3.0:1789760752.738371:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5022 00000100:00100000:3.0:1789760752.738383:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733937897717 time 734001189617 state 0x2 ref 0 00000100:00100000:3.0:1789760752.741408:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.742299:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271168 00000100:00100000:1.0:1789760752.742313:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271168 00000100:00100000:1.0:1789760752.742388:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 733937897717 time 734001189617 state 0x0 ref 1 00000100:00100000:1.0:1789760752.742406:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734001450033 time 734001189617 state 0x1 ref 1 00000100:00100000:1.0:1789760752.742493:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734001450033 time 734057794584 state 0x0 ref 1 00000100:00100000:1.0:1789760752.742509:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.742573:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20451328, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.742589:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.742609:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760752.742614:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.742622:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.742638:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.803519:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972313 is committed 00002000:00010000:3.0:1789760752.803735:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20451328 -> 20455424 00010000:00010000:3.0:1789760752.803756:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803781:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803792:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803805:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.803811:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803831:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803858:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.803866:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b84c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.805094:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62574us (63687us total) trans 4294972313 rc 0/0 00000100:00100000:3.0:1789760752.805111:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5023 00000100:00100000:3.0:1789760752.805124:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734001450033 time 734057794584 state 0x2 ref 0 00000100:00100000:1.0:1789760752.810457:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760752.810548:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271296 00000100:00100000:2.0:1789760752.810562:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271296 00000100:00100000:2.0:1789760752.810636:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734001450033 time 734057794584 state 0x0 ref 1 00000100:00100000:2.0:1789760752.810655:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734058055000 time 734057794584 state 0x1 ref 1 00000100:00100000:2.0:1789760752.810737:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734058055000 time 734125994736 state 0x0 ref 1 00000100:00100000:2.0:1789760752.810754:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507271296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760752.810833:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20455424, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:2.0:1789760752.810845:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.810864:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760752.810869:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.810876:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.810893:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.864048:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20455424 -> 20459520 00010000:00010000:3.0:1789760752.864075:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864090:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864108:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864116:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.864122:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864132:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864157:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.864165:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b84c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.864290:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507271296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53533us (53841us total) trans 4294972314 rc 0/0 00000100:00100000:3.0:1789760752.864301:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5024 00000100:00100000:3.0:1789760752.864314:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734058055000 time 734125994736 state 0x2 ref 0 00000001:00080000:0.0:1789760752.864396:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972314 is committed 00000100:00100000:3.0:1789760752.867952:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.868167:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271424 00000100:00100000:1.0:1789760752.868181:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271424 00000100:00100000:1.0:1789760752.868268:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734058055000 time 734125994736 state 0x0 ref 1 00000100:00100000:1.0:1789760752.868286:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734126255152 time 734125994736 state 0x1 ref 1 00000100:00100000:1.0:1789760752.868328:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734126255152 time 734183631063 state 0x0 ref 1 00000100:00100000:1.0:1789760752.868345:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507271424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.868410:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20459520, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.868421:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.868442:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760752.868447:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.868455:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.868471:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.923384:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20459520 -> 20463616 00010000:00010000:3.0:1789760752.923411:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923442:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923452:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923459:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760752.923465:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923475:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923500:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923508:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b84d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.923647:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507271424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55299us (55705us total) trans 4294972315 rc 0/0 00000100:00100000:3.0:1789760752.923660:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5025 00000100:00100000:3.0:1789760752.923672:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734126255152 time 734183631063 state 0x2 ref 0 00000001:00080000:0.0:1789760752.924080:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972315 is committed 00000100:00100000:3.0:1789760752.934144:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760752.934893:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271552 00000100:00100000:1.0:1789760752.934908:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271552 00000100:00100000:1.0:1789760752.934990:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734126255152 time 734183631063 state 0x0 ref 1 00000100:00100000:1.0:1789760752.935012:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734183891479 time 734183631063 state 0x1 ref 1 00000100:00100000:1.0:1789760752.935049:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734183891479 time 734250351521 state 0x0 ref 1 00000100:00100000:1.0:1789760752.935063:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507271552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760752.935126:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20463616, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:1.0:1789760752.935137:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.935157:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760752.935162:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.935169:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.935186:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.975960:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972316 is committed 00002000:00010000:1.0:1789760752.976110:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20463616 -> 20467712 00010000:00010000:1.0:1789760752.976128:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976143:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976225:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976235:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760752.976240:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976251:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976276:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.976284:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b84d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.976445:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507271552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41378us (42308us total) trans 4294972316 rc 0/0 00000100:00100000:1.0:1789760752.976458:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5026 00000100:00100000:1.0:1789760752.976470:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734183891479 time 734250351521 state 0x2 ref 0 00000100:00100000:1.0:1789760752.979642:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760752.979880:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271680 00000100:00100000:3.0:1789760752.979894:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271680 00000100:00100000:3.0:1789760752.979969:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734183891479 time 734250351521 state 0x0 ref 1 00000100:00100000:3.0:1789760752.979988:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734250611937 time 734250351521 state 0x1 ref 1 00000100:00100000:3.0:1789760752.980075:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734250611937 time 734295376805 state 0x0 ref 1 00000100:00100000:3.0:1789760752.980090:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507271680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760752.980161:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20467712, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:3.0:1789760752.980174:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.980194:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760752.980200:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.980207:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.980224:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.016365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972317 is committed 00002000:00010000:3.0:1789760753.016523:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20467712 -> 20471808 00010000:00010000:3.0:1789760753.016541:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016554:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016565:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016571:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.016575:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016582:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016601:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.016606:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.016918:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507271680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36824us (37283us total) trans 4294972317 rc 0/0 00000100:00100000:3.0:1789760753.016930:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5027 00000100:00100000:3.0:1789760753.016943:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734250611937 time 734295376805 state 0x2 ref 0 00000100:00100000:2.0:1789760753.023483:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.024496:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271808 00000100:00100000:3.0:1789760753.024512:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271808 00000100:00100000:3.0:1789760753.024610:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734250611937 time 734295376805 state 0x0 ref 1 00000100:00100000:3.0:1789760753.024629:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734295637221 time 734295376805 state 0x1 ref 1 00000100:00080000:0.0:1789760753.024746:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760753.024760:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734295637221 time 734340062904 state 0x0 ref 1 00000100:00100000:3.0:1789760753.024775:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.024836:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20471808, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa007 00010000:00010000:3.0:1789760753.024846:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.024865:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760753.024870:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.024878:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.024894:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.066999:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972318 is committed 00002000:00010000:0.0:1789760753.067196:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20471808 -> 20475904 00002000:00010000:0.0:1789760753.067201:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:0.0:1789760753.067204:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:0.0:1789760753.067221:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067237:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067248:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067262:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760753.067268:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067278:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067304:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.067312:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b84e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.067519:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507271808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42739us (44053us total) trans 4294972318 rc 0/0 00000100:00100000:0.0:1789760753.067532:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5028 00000100:00100000:0.0:1789760753.067545:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734295637221 time 734340062904 state 0x2 ref 0 00000100:00100000:0.0:1789760753.078575:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760753.082062:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507271936 00000100:00100000:1.0:1789760753.083549:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507271936 00000100:00100000:2.0:1789760753.084627:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734295637221 time 734340062904 state 0x0 ref 1 00000100:00100000:2.0:1789760753.084651:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734340323320 time 734340062904 state 0x1 ref 1 00000100:00100000:2.0:1789760753.084749:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734340323320 time 734400051861 state 0x0 ref 1 00000100:00100000:2.0:1789760753.084766:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507271936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760753.085156:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20475904, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:2.0:1789760753.085170:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.085190:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760753.085195:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.085202:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.085218:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.189181:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972319 is committed 00002000:00010000:1.0:1789760753.189312:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:1.0:1789760753.189333:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189356:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189367:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189374:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760753.189379:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189389:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189411:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.189418:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b84ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.189537:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507271936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 104769us (110970us total) trans 4294972319 rc 0/0 00000100:00100000:1.0:1789760753.189550:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5029 00000100:00100000:1.0:1789760753.189563:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734340323320 time 734400051861 state 0x2 ref 0 00000100:00100000:1.0:1789760753.195960:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.196034:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272064 00000100:00100000:3.0:1789760753.196047:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272064 00000100:00100000:3.0:1789760753.196117:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734340323320 time 734400051861 state 0x0 ref 1 00000100:00100000:3.0:1789760753.196135:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734400312277 time 734400051861 state 0x1 ref 1 00000100:00100000:3.0:1789760753.196685:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734400312277 time 734511984972 state 0x0 ref 1 00000100:00100000:3.0:1789760753.196846:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507272064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760753.196990:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20480000, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:2.0:1789760753.197004:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.197023:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760753.197028:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.197035:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.197051:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.301357:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:1.0:1789760753.301386:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301402:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301412:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301420:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760753.301425:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301435:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301457:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.301465:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b84f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.301588:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507272064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 104845us (105633us total) trans 4294972320 rc 0/0 00000100:00100000:1.0:1789760753.301602:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5030 00000100:00100000:1.0:1789760753.301620:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734400312277 time 734511984972 state 0x2 ref 0 00000001:00080000:3.0:1789760753.304117:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00000100:00100000:2.0:1789760753.304335:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760753.304434:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272192 00000100:00100000:0.0:1789760753.304446:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272192 00000100:00100000:0.0:1789760753.304511:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734400312277 time 734511984972 state 0x0 ref 1 00000100:00100000:0.0:1789760753.304529:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734512245388 time 734511984972 state 0x1 ref 1 00000100:00100000:0.0:1789760753.304593:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734512245388 time 734619895637 state 0x0 ref 1 00000100:00100000:0.0:1789760753.304607:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507272192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760753.304665:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20484096, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:0.0:1789760753.304675:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.304736:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760753.304742:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.304750:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.304767:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.342878:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972321 is committed 00002000:00010000:0.0:1789760753.342990:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20484096 -> 20488192 00010000:00010000:0.0:1789760753.343009:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343025:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343035:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343044:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760753.343051:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343062:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343087:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.343094:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b84fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.343309:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507272192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38696us (38972us total) trans 4294972321 rc 0/0 00000100:00100000:0.0:1789760753.343322:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5031 00000100:00100000:0.0:1789760753.343335:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734512245388 time 734619895637 state 0x2 ref 0 00000100:00100000:2.0:1789760753.346728:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.347491:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272320 00000100:00100000:3.0:1789760753.347507:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272320 00000100:00100000:3.0:1789760753.347582:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734512245388 time 734619895637 state 0x0 ref 1 00000100:00100000:3.0:1789760753.347600:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734620156053 time 734619895637 state 0x1 ref 1 00000100:00100000:3.0:1789760753.347633:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734620156053 time 734662936096 state 0x0 ref 1 00000100:00100000:3.0:1789760753.347647:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507272320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.347752:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20488192, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.347764:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.347783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760753.347788:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.347795:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.347813:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.368086:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.368125:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.368162:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.368181:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.368200:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.368234:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272704 00000100:00100000:3.0:1789760753.368246:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272704 00000100:00100000:3.0:1789760753.368308:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272960 00000100:00100000:3.0:1789760753.368311:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272960 00000100:00100000:3.0:1789760753.368326:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272576 00000100:00100000:3.0:1789760753.368328:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272576 00000100:00100000:3.0:1789760753.368340:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272832 00000100:00100000:3.0:1789760753.368342:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272832 00000100:00100000:3.0:1789760753.368355:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 908 00000100:00100000:3.0:1789760753.368365:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507272704:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760753.368444:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507272704:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 80us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.368452:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 908 00000100:00100000:3.0:1789760753.368468:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 909 00000100:00100000:3.0:1789760753.368473:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507272960:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760753.368493:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507272960:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 20us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.368499:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 909 00000100:00100000:3.0:1789760753.368508:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 910 00000100:00100000:3.0:1789760753.368513:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507272576:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760753.368534:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507272576:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 20us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.368540:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 910 00000100:00100000:3.0:1789760753.368552:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 911 00000100:00100000:3.0:1789760753.368559:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507272832:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760753.368582:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507272832:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.368588:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 911 00000100:00080000:3.0:1789760753.368612:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760753.368617:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760753.368727:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507272448 00000100:00100000:2.0:1789760753.368732:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507272448 00000100:00100000:2.0:1789760753.368772:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 472 00000100:00100000:2.0:1789760753.368779:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507272448:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760753.368805:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507272448:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 28us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760753.368812:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 472 00000001:00080000:2.0:1789760753.380034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972322 is committed 00002000:00010000:0.0:1789760753.380153:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20488192 -> 20492288 00010000:00010000:0.0:1789760753.380173:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380189:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380199:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380206:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760753.380214:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380224:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380247:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.380255:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8501 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.380371:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507272320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32719us (33684us total) trans 4294972322 rc 0/0 00000100:00100000:0.0:1789760753.380383:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5032 00000100:00100000:0.0:1789760753.380395:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734620156053 time 734662936096 state 0x2 ref 0 00000100:00100000:2.0:1789760753.384095:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760753.384263:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273088 00000100:00100000:1.0:1789760753.384275:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273088 00000100:00100000:1.0:1789760753.384343:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734620156053 time 734662936096 state 0x0 ref 1 00000100:00100000:1.0:1789760753.384361:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734663196512 time 734662936096 state 0x1 ref 1 00000100:00100000:1.0:1789760753.384425:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734663196512 time 734699727774 state 0x0 ref 1 00000100:00100000:1.0:1789760753.384439:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760753.384500:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20492288, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:1.0:1789760753.384511:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.384530:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760753.384535:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.384542:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.384559:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.421943:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972323 is committed 00002000:00010000:1.0:1789760753.422093:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20492288 -> 20496384 00010000:00010000:1.0:1789760753.422114:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422131:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422206:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422217:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760753.422223:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422234:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422258:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.422268:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8508 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.422424:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37979us (38336us total) trans 4294972323 rc 0/0 00000100:00100000:1.0:1789760753.422436:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5033 00000100:00100000:1.0:1789760753.422448:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734663196512 time 734699727774 state 0x2 ref 0 00000100:00100000:3.0:1789760753.425436:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760753.425574:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273216 00000100:00100000:0.0:1789760753.425586:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273216 00000100:00100000:0.0:1789760753.425651:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734663196512 time 734699727774 state 0x0 ref 1 00000100:00100000:0.0:1789760753.425669:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734699988190 time 734699727774 state 0x1 ref 1 00000100:00100000:0.0:1789760753.425830:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734699988190 time 734741131888 state 0x0 ref 1 00000100:00100000:0.0:1789760753.425845:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507273216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760753.425906:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20496384, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:0.0:1789760753.425916:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.425935:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760753.425940:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.425946:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.425961:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.455946:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972324 is committed 00002000:00010000:2.0:1789760753.456085:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20496384 -> 20500480 00010000:00010000:2.0:1789760753.456105:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456121:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456131:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456139:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760753.456205:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456218:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456244:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.456259:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b850f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.456405:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507273216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30556us (30976us total) trans 4294972324 rc 0/0 00000100:00100000:2.0:1789760753.456417:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5034 00000100:00100000:2.0:1789760753.456429:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734699988190 time 734741131888 state 0x2 ref 0 00000100:00100000:2.0:1789760753.459633:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.459725:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273344 00000100:00100000:3.0:1789760753.459746:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273344 00000100:00100000:3.0:1789760753.459813:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734699988190 time 734741131888 state 0x0 ref 1 00000100:00100000:3.0:1789760753.459833:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734741392304 time 734741131888 state 0x1 ref 1 00000100:00100000:3.0:1789760753.459869:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734741392304 time 734775172295 state 0x0 ref 1 00000100:00100000:3.0:1789760753.459883:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.459942:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20500480, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.459954:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.459973:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760753.459979:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.459986:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.460003:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.486477:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20500480 -> 20504576 00010000:00010000:3.0:1789760753.486505:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486521:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486532:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486540:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.486547:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486557:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486581:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.486589:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8516 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.486754:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507273344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26866us (27128us total) trans 4294972325 rc 0/0 00000100:00100000:3.0:1789760753.486770:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5035 00000100:00100000:3.0:1789760753.486782:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734741392304 time 734775172295 state 0x2 ref 0 00000100:00100000:3.0:1789760753.489358:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760753.489496:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273472 00000100:00100000:1.0:1789760753.489509:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273472 00000100:00100000:1.0:1789760753.489584:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734741392304 time 734775172295 state 0x0 ref 1 00000100:00100000:1.0:1789760753.489603:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734775432711 time 734775172295 state 0x1 ref 1 00000100:00100000:1.0:1789760753.489674:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734775432711 time 734804976684 state 0x0 ref 1 00000100:00100000:1.0:1789760753.489689:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507273472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760753.489799:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20504576, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:1.0:1789760753.489810:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.489829:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760753.489834:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.489841:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.489858:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.490417:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972325 is committed 00000001:00080000:3.0:1789760753.519521:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972326 is committed 00002000:00010000:2.0:1789760753.519642:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20504576 -> 20508672 00010000:00010000:2.0:1789760753.519663:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519679:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519689:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519739:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760753.519746:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519757:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519790:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.519799:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b851d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.519927:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507273472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30233us (30575us total) trans 4294972326 rc 0/0 00000100:00100000:2.0:1789760753.519939:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5036 00000100:00100000:2.0:1789760753.519951:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734775432711 time 734804976684 state 0x2 ref 0 00000100:00100000:2.0:1789760753.523246:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760753.524271:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273600 00000100:00100000:0.0:1789760753.524287:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273600 00000100:00100000:0.0:1789760753.524370:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734775432711 time 734804976684 state 0x0 ref 1 00000100:00100000:0.0:1789760753.524388:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734805237100 time 734804976684 state 0x1 ref 1 00000100:00100000:0.0:1789760753.524425:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734805237100 time 734839728505 state 0x0 ref 1 00000100:00100000:0.0:1789760753.524439:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760753.524505:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20508672, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:0.0:1789760753.524517:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.524539:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760753.524545:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.524554:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.524574:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.561606:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20508672 -> 20512768 00010000:00010000:3.0:1789760753.561633:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.561649:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.561660:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.561668:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.561673:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.561683:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.561839:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972327 is committed 00010000:00010000:3.0:1789760753.561899:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.561908:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8524 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.562042:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37589us (38795us total) trans 4294972327 rc 0/0 00000100:00100000:3.0:1789760753.562054:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5037 00000100:00100000:3.0:1789760753.562066:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734805237100 time 734839728505 state 0x2 ref 0 00010000:00010000:3.0:1789760753.564160:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760753.565044:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760753.565156:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273728 00000100:00100000:2.0:1789760753.565169:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273728 00000100:00100000:2.0:1789760753.565246:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734805237100 time 734839728505 state 0x0 ref 1 00000100:00100000:2.0:1789760753.565265:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734839988921 time 734839728505 state 0x1 ref 1 00000100:00100000:2.0:1789760753.565300:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734839988921 time 734880602221 state 0x0 ref 1 00000100:00100000:2.0:1789760753.565315:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507273728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760753.565382:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20512768, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:2.0:1789760753.565393:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.565418:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760753.565424:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.565431:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.565448:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.598981:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20512768 -> 20516864 00010000:00010000:2.0:1789760753.599008:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599025:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599035:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599044:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760753.599050:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599060:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599088:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.599097:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b852b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.599223:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972328 is committed 00000100:00100000:2.0:1789760753.599397:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507273728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34078us (34355us total) trans 4294972328 rc 0/0 00000100:00100000:2.0:1789760753.599409:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5038 00000100:00100000:2.0:1789760753.599422:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734839988921 time 734880602221 state 0x2 ref 0 00000100:00100000:2.0:1789760753.603292:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760753.603429:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273856 00000100:00100000:1.0:1789760753.603444:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273856 00000100:00100000:1.0:1789760753.603521:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734839988921 time 734880602221 state 0x0 ref 1 00000100:00100000:1.0:1789760753.603542:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734880862637 time 734880602221 state 0x1 ref 1 00000100:00100000:1.0:1789760753.603615:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734880862637 time 734918917019 state 0x0 ref 1 00000100:00100000:1.0:1789760753.603632:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760753.603734:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20516864, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:1.0:1789760753.603757:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.603778:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760753.603784:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.603792:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.603810:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.634835:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972329 is committed 00002000:00010000:1.0:1789760753.635090:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20516864 -> 20520960 00010000:00010000:1.0:1789760753.635111:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635127:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635139:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635159:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760753.635166:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635178:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635212:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.635222:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8532 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.635353:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31717us (32070us total) trans 4294972329 rc 0/0 00000100:00100000:1.0:1789760753.635365:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5039 00000100:00100000:1.0:1789760753.635378:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734880862637 time 734918917019 state 0x2 ref 0 00000100:00100000:2.0:1789760753.638288:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.638450:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507273984 00000100:00100000:3.0:1789760753.638464:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507273984 00000100:00100000:3.0:1789760753.638538:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734880862637 time 734918917019 state 0x0 ref 1 00000100:00100000:3.0:1789760753.638556:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734919177435 time 734918917019 state 0x1 ref 1 00000100:00100000:3.0:1789760753.638591:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734919177435 time 734953893740 state 0x0 ref 1 00000100:00100000:3.0:1789760753.638606:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.638673:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20520960, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.638684:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.638767:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760753.638773:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.638781:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.638800:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.673169:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972330 is committed 00002000:00010000:0.0:1789760753.673358:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20520960 -> 20525056 00010000:00010000:0.0:1789760753.673379:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673395:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673405:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673413:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760753.673419:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673430:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673457:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.673465:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8539 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.673596:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507273984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34981us (35312us total) trans 4294972330 rc 0/0 00000100:00100000:0.0:1789760753.673608:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5040 00000100:00100000:0.0:1789760753.673620:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734919177435 time 734953893740 state 0x2 ref 0 00000100:00100000:3.0:1789760753.676409:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760753.676980:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274112 00000100:00100000:2.0:1789760753.676995:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274112 00000100:00100000:2.0:1789760753.677079:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734919177435 time 734953893740 state 0x0 ref 1 00000100:00100000:2.0:1789760753.677100:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734954154156 time 734953893740 state 0x1 ref 1 00000100:00100000:2.0:1789760753.677243:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734954154156 time 734992544856 state 0x0 ref 1 00000100:00100000:2.0:1789760753.677258:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507274112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760753.677321:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20525056, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:2.0:1789760753.677331:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.677350:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760753.677356:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.677363:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.677379:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.706654:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972331 is committed 00002000:00010000:3.0:1789760753.706755:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20525056 -> 20529152 00010000:00010000:3.0:1789760753.706777:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706791:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706801:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706809:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.706815:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706825:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706854:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.706862:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8540 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.706987:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507274112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29721us (30589us total) trans 4294972331 rc 0/0 00000100:00100000:3.0:1789760753.707001:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5041 00000100:00100000:3.0:1789760753.707013:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734954154156 time 734992544856 state 0x2 ref 0 00000100:00100000:2.0:1789760753.708851:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760753.709251:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274240 00000100:00100000:1.0:1789760753.709264:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274240 00000100:00100000:1.0:1789760753.709336:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734954154156 time 734992544856 state 0x0 ref 1 00000100:00100000:1.0:1789760753.709355:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734992805272 time 734992544856 state 0x1 ref 1 00000100:00100000:1.0:1789760753.709513:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734992805272 time 735024814098 state 0x0 ref 1 00000100:00100000:1.0:1789760753.709529:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507274240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760753.709590:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20529152, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:1.0:1789760753.709600:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.709620:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760753.709626:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.709633:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.709649:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.755326:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972332 is committed 00002000:00010000:3.0:1789760753.755523:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20529152 -> 20533248 00010000:00010000:3.0:1789760753.755544:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755561:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755572:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755580:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.755585:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755595:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755622:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.755635:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8547 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.755805:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507274240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46272us (46958us total) trans 4294972332 rc 0/0 00000100:00100000:3.0:1789760753.755816:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5042 00000100:00100000:3.0:1789760753.755828:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734992805272 time 735024814098 state 0x2 ref 0 00000100:00100000:2.0:1789760753.761035:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760753.761277:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274368 00000100:00100000:0.0:1789760753.761291:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274368 00000100:00100000:0.0:1789760753.761364:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 734992805272 time 735024814098 state 0x0 ref 1 00000100:00100000:0.0:1789760753.761383:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735025074514 time 735024814098 state 0x1 ref 1 00000100:00100000:0.0:1789760753.761418:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735025074514 time 735076721007 state 0x0 ref 1 00000100:00100000:0.0:1789760753.761432:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760753.761492:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20533248, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:0.0:1789760753.761504:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.761524:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760753.761530:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.761537:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.761555:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.807319:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972333 is committed 00002000:00010000:3.0:1789760753.807525:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20533248 -> 20537344 00010000:00010000:3.0:1789760753.807546:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807562:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807577:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807586:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.807592:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807602:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807632:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.807641:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b854e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.807988:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46550us (46958us total) trans 4294972333 rc 0/0 00000100:00100000:3.0:1789760753.808001:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5043 00000100:00100000:3.0:1789760753.808013:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735025074514 time 735076721007 state 0x2 ref 0 00000100:00100000:3.0:1789760753.811038:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760753.811828:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274496 00000100:00100000:0.0:1789760753.811842:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274496 00000100:00100000:1.0:1789760753.812167:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735025074514 time 735076721007 state 0x0 ref 1 00000100:00100000:1.0:1789760753.812187:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735076981423 time 735076721007 state 0x1 ref 1 00000100:00100000:1.0:1789760753.812291:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735076981423 time 735127592401 state 0x0 ref 1 00000100:00100000:1.0:1789760753.812307:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507274496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760753.812375:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20537344, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:1.0:1789760753.812386:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.812407:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760753.812412:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.812420:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.812438:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.850518:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00002000:00010000:3.0:1789760753.850641:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20537344 -> 20541440 00010000:00010000:3.0:1789760753.850662:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850677:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850688:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850725:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.850740:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850758:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850785:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.850793:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8555 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.850921:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507274496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38610us (39890us total) trans 4294972334 rc 0/0 00000100:00100000:3.0:1789760753.850935:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5044 00000100:00100000:3.0:1789760753.850948:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735076981423 time 735127592401 state 0x2 ref 0 00000100:00100000:2.0:1789760753.853503:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.853590:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274624 00000100:00100000:3.0:1789760753.853602:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274624 00000100:00100000:3.0:1789760753.853670:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735076981423 time 735127592401 state 0x0 ref 1 00000100:00100000:3.0:1789760753.853687:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735127852817 time 735127592401 state 0x1 ref 1 00000100:00100000:3.0:1789760753.853922:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735127852817 time 735169223133 state 0x0 ref 1 00000100:00100000:3.0:1789760753.853937:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.854001:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20541440, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.854013:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.854138:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760753.854144:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.854152:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.854171:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.883840:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760753.883874:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108023480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538278400:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760753.883977:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.884184:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278400 00000100:00100000:0.0:1789760753.884194:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278400 00000100:00100000:0.0:1789760753.884265:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1789760753.884276:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538278400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760753.884319:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760753.884373:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538278400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760753.884381:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00080000:0.0:1789760753.884405:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760753.884422:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538278528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760753.884448:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.884501:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108023480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538278400:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760753.884622:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278528 00000100:00100000:3.0:1789760753.884628:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278528 00000100:00100000:3.0:1789760753.884676:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:3.0:1789760753.884683:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538278528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760753.884723:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760753.884762:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538278528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.884768:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:3.0:1789760753.884905:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538278656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.884943:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.884968:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538279168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.884995:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.885020:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538279680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.885047:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.885069:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760753.885074:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538278784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.885102:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.885121:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538279296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.885143:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.885160:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538279808:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.885181:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.885217:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538278912:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.885243:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.885810:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278656 00000100:00100000:0.0:1789760753.885819:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278656 00000100:00100000:0.0:1789760753.885862:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789760753.885870:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538278656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.885927:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538278656:12345-0@lo:400:kworker.0 Request processed in 57us (982us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760753.885934:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00080000:0.0:1789760753.885970:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760753.885999:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538278656:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886243:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538279040:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886278:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.886310:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538279552:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886335:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.886400:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279680 00000100:00100000:0.0:1789760753.886405:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279680 00000100:00100000:0.0:1789760753.886441:0:8251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279552 00000100:00100000:0.0:1789760753.886444:0:8251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279552 00000100:00100000:0.0:1789760753.886465:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1789760753.886472:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538279680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886513:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538279680:12345-0@lo:400:kworker.0 Request processed in 41us (1464us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.886519:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538279424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886520:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1789760753.886533:0:8251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1789760753.886538:0:8251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538279552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.886547:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760753.886565:0:8251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538279552:12345-0@lo:400:kworker.0 Request processed in 27us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.886565:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538279936:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886572:0:8251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:3.0:1789760753.886586:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789760753.886594:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760753.886621:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278784 00000100:00100000:3.0:1789760753.886625:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278784 00000100:00100000:3.0:1789760753.886647:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279296 00000100:00100000:3.0:1789760753.886651:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279296 00000100:00080000:0.0:1789760753.886665:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760753.886670:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279808 00000100:00100000:3.0:1789760753.886673:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279808 00000100:00100000:0.0:1789760753.886686:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538279680:0@lo:400:kworker.0 00000100:00080000:0.0:1789760753.886729:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760753.886745:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538279552:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.886787:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538278528:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760753.886873:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279040 00000100:00100000:3.0:1789760753.886878:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279040 00000100:00100000:3.0:1789760753.886894:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279424 00000100:00100000:3.0:1789760753.886896:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279424 00000100:00080000:0.0:1789760753.886916:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760753.886919:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279936 00000100:00100000:3.0:1789760753.886922:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279936 00000100:00080000:0.0:1789760753.886939:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760753.886939:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 912 00000100:00100000:3.0:1789760753.886946:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538278784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887001:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538278784:12345-0@lo:400:kworker.0 Request processed in 55us (1898us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.887011:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 912 00000100:00100000:3.0:1789760753.887024:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:3.0:1789760753.887030:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538279296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887060:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538279296:12345-0@lo:400:kworker.0 Request processed in 30us (1917us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.887066:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:3.0:1789760753.887079:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:3.0:1789760753.887085:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538279808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887114:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538279808:12345-0@lo:400:kworker.0 Request processed in 29us (1931us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760753.887117:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538278784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887120:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:0.0:1789760753.887128:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538279296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887131:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 915 00000100:00100000:3.0:1789760753.887137:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538279040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.887139:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538279808:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887167:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538279040:12345-0@lo:400:kworker.0 Request processed in 30us (889us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.887173:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 915 00000100:00100000:0.0:1789760753.887176:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f41f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538279040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887184:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 916 00000100:00100000:3.0:1789760753.887190:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538279424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760753.887277:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 917 00000100:00100000:2.0:1789760753.887288:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538279936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760753.887337:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538279936:12345-0@lo:400:kworker.0 Request processed in 53us (750us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760753.887344:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 917 00000100:00100000:3.0:1789760753.887352:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538279424:12345-0@lo:400:kworker.0 Request processed in 162us (804us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.887358:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 916 00000100:00100000:0.0:1789760753.887404:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108017480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538279424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.887414:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538279936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760753.887467:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538279168 00000100:00100000:2.0:1789760753.887471:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538279168 00000100:00100000:2.0:1789760753.887538:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538278912 00000100:00100000:2.0:1789760753.887544:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538278912 00000100:00100000:0.0:1789760753.887572:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1789760753.887577:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:0.0:1789760753.887579:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538279168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760753.887586:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538278912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760753.887620:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538279168:12345-0@lo:400:kworker.0 Request processed in 42us (2624us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760753.887626:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1789760753.887628:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538278912:12345-0@lo:400:kworker.0 Request processed in 44us (2385us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760753.887633:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:1.0:1789760753.887912:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108014380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538279168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.887957:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538278912:0@lo:400:kworker.0 00000001:00080000:0.0:1789760753.888775:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972335 is committed 00002000:00010000:3.0:1789760753.889093:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20541440 -> 20545536 00010000:00010000:3.0:1789760753.889112:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889128:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889137:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889146:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.889151:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889161:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889187:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889196:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b855c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.889515:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35574us (36023us total) trans 4294972335 rc 0/0 00000100:00100000:3.0:1789760753.889527:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5045 00000100:00100000:3.0:1789760753.889539:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735127852817 time 735169223133 state 0x2 ref 0 00000100:00100000:2.0:1789760753.892679:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.892883:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274752 00000100:00100000:3.0:1789760753.892896:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274752 00000100:00100000:3.0:1789760753.892968:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735127852817 time 735169223133 state 0x0 ref 1 00000100:00100000:3.0:1789760753.892986:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735169483549 time 735169223133 state 0x1 ref 1 00000100:00100000:3.0:1789760753.893360:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735169483549 time 735208660047 state 0x0 ref 1 00000100:00100000:3.0:1789760753.893378:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507274752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.893499:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20545536, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.893511:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.893530:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760753.893537:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.893544:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.893561:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.940685:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972336 is committed 00002000:00010000:0.0:1789760753.940932:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20545536 -> 20549632 00010000:00010000:0.0:1789760753.940954:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.940970:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.940980:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.940992:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760753.940998:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.941009:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.941100:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.941110:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8563 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.941304:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507274752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47921us (48633us total) trans 4294972336 rc 0/0 00000100:00100000:0.0:1789760753.941317:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5046 00000100:00100000:0.0:1789760753.941329:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735169483549 time 735208660047 state 0x2 ref 0 00000100:00100000:0.0:1789760753.945209:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.945308:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507274880 00000100:00100000:3.0:1789760753.945322:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507274880 00000100:00100000:3.0:1789760753.945396:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735169483549 time 735208660047 state 0x0 ref 1 00000100:00100000:3.0:1789760753.945416:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735208920463 time 735208660047 state 0x1 ref 1 00000100:00100000:3.0:1789760753.945503:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735208920463 time 735260805434 state 0x0 ref 1 00000100:00100000:3.0:1789760753.945521:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.945585:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20549632, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.945596:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.945617:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760753.945622:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.945629:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.945647:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.977572:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972337 is committed 00002000:00010000:3.0:1789760753.977792:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20549632 -> 20553728 00010000:00010000:3.0:1789760753.977814:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977828:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977839:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977847:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760753.977853:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977863:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977888:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.977896:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b856a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.979340:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507274880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32497us (32822us total) trans 4294972337 rc 0/0 00000100:00100000:3.0:1789760753.979359:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5047 00000100:00100000:3.0:1789760753.979387:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735208920463 time 735260805434 state 0x2 ref 0 00000100:00100000:2.0:1789760753.985330:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760753.985567:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275008 00000100:00100000:3.0:1789760753.985582:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275008 00000100:00100000:3.0:1789760753.985658:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735208920463 time 735260805434 state 0x0 ref 1 00000100:00100000:3.0:1789760753.985676:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735261065850 time 735260805434 state 0x1 ref 1 00000100:00100000:3.0:1789760753.985806:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735261065850 time 735301108717 state 0x0 ref 1 00000100:00100000:3.0:1789760753.985821:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507275008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760753.985888:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20553728, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:3.0:1789760753.985899:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.985919:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760753.985925:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.985933:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.985948:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.017438:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20553728 -> 20557824 00010000:00010000:2.0:1789760754.017464:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017486:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017496:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017504:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.017509:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017520:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017544:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.017552:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8571 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.017680:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507275008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31853us (32357us total) trans 4294972338 rc 0/0 00000100:00100000:2.0:1789760754.017745:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5048 00000100:00100000:2.0:1789760754.017758:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735261065850 time 735301108717 state 0x2 ref 0 00000001:00080000:0.0:1789760754.018471:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972338 is committed 00000100:00100000:2.0:1789760754.021169:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760754.021316:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275136 00000100:00100000:0.0:1789760754.021329:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275136 00000100:00100000:0.0:1789760754.021406:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735261065850 time 735301108717 state 0x0 ref 1 00000100:00100000:0.0:1789760754.021426:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735301369133 time 735301108717 state 0x1 ref 1 00000100:00100000:0.0:1789760754.021467:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735301369133 time 735336770433 state 0x0 ref 1 00000100:00100000:0.0:1789760754.021481:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507275136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760754.021548:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20557824, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa007 00010000:00010000:0.0:1789760754.021559:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.021578:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760754.021582:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.021590:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.021608:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.057334:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00002000:00010000:1.0:1789760754.057432:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20557824 -> 20561920 00002000:00010000:1.0:1789760754.057437:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:1.0:1789760754.057440:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:1.0:1789760754.057458:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057474:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057484:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057491:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760754.057497:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057508:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057533:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057542:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8578 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.057680:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507275136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36194us (36518us total) trans 4294972339 rc 0/0 00000100:00100000:1.0:1789760754.057690:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5049 00000100:00100000:1.0:1789760754.057852:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735301369133 time 735336770433 state 0x2 ref 0 00000100:00100000:1.0:1789760754.064029:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.064201:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275264 00000100:00100000:3.0:1789760754.064215:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275264 00000100:00100000:3.0:1789760754.064295:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735301369133 time 735336770433 state 0x0 ref 1 00000100:00100000:3.0:1789760754.064315:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735337030849 time 735336770433 state 0x1 ref 1 00000100:00100000:3.0:1789760754.064352:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735337030849 time 735379654231 state 0x0 ref 1 00000100:00100000:3.0:1789760754.064366:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.064441:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20561920, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.064453:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.064473:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760754.064480:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.064488:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.064510:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.112843:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972340 is committed 00002000:00010000:0.0:1789760754.113146:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20561920 -> 20566016 00010000:00010000:0.0:1789760754.113166:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113181:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113191:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113199:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760754.113206:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113217:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113243:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.113251:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b857f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.113377:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49006us (49357us total) trans 4294972340 rc 0/0 00000100:00100000:0.0:1789760754.113390:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5050 00000100:00100000:0.0:1789760754.113402:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735337030849 time 735379654231 state 0x2 ref 0 00000100:00100000:3.0:1789760754.118598:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760754.119413:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275392 00000100:00100000:2.0:1789760754.119428:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275392 00000100:00100000:2.0:1789760754.119633:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735337030849 time 735379654231 state 0x0 ref 1 00000100:00100000:2.0:1789760754.119657:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735379914647 time 735379654231 state 0x1 ref 1 00000100:00100000:2.0:1789760754.119744:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735379914647 time 735435042620 state 0x0 ref 1 00000100:00100000:2.0:1789760754.120007:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760754.120078:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20566016, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:2.0:1789760754.120089:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.120108:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760754.120112:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.120119:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.120135:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.202129:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972341 is committed 00002000:00010000:3.0:1789760754.202203:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20566016 -> 20570112 00010000:00010000:3.0:1789760754.202223:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202238:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202248:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202256:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.202262:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202272:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202299:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.202308:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8586 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.202437:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82671us (83846us total) trans 4294972341 rc 0/0 00000100:00100000:3.0:1789760754.202448:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5051 00000100:00100000:3.0:1789760754.202460:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735379914647 time 735435042620 state 0x2 ref 0 00000100:00100000:2.0:1789760754.209018:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760754.209129:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275520 00000100:00100000:0.0:1789760754.209141:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275520 00000100:00100000:0.0:1789760754.209216:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735379914647 time 735435042620 state 0x0 ref 1 00000100:00100000:0.0:1789760754.209236:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735435303036 time 735435042620 state 0x1 ref 1 00000100:00100000:0.0:1789760754.209279:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735435303036 time 735524581987 state 0x0 ref 1 00000100:00100000:0.0:1789760754.209294:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760754.209364:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20570112, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:0.0:1789760754.209376:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.209397:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760754.209403:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.209410:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.209429:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.272532:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00002000:00010000:0.0:1789760754.272671:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20570112 -> 20574208 00010000:00010000:0.0:1789760754.272746:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272762:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272773:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272781:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760754.272787:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272799:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272827:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.272835:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b858d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.273578:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64274us (64562us total) trans 4294972342 rc 0/0 00000100:00100000:1.0:1789760754.273595:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5052 00000100:00100000:1.0:1789760754.273609:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735435303036 time 735524581987 state 0x2 ref 0 00000100:00100000:1.0:1789760754.280496:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760754.280986:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275648 00000100:00100000:0.0:1789760754.281002:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275648 00000100:00100000:0.0:1789760754.281082:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735435303036 time 735524581987 state 0x0 ref 1 00000100:00100000:0.0:1789760754.281102:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735524842403 time 735524581987 state 0x1 ref 1 00000100:00100000:0.0:1789760754.281137:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735524842403 time 735596440264 state 0x0 ref 1 00000100:00100000:0.0:1789760754.281152:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760754.281221:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20574208, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:0.0:1789760754.281231:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.281252:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760754.281257:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.281264:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.281293:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.334775:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00002000:00010000:0.0:1789760754.335637:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20574208 -> 20578304 00010000:00010000:0.0:1789760754.335660:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335675:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335686:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335870:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760754.335877:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335888:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335917:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.335926:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8594 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.336213:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55054us (55725us total) trans 4294972343 rc 0/0 00000100:00100000:1.0:1789760754.336226:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5053 00000100:00100000:1.0:1789760754.336239:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735524842403 time 735596440264 state 0x2 ref 0 00000100:00100000:3.0:1789760754.339642:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760754.340042:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275776 00000100:00100000:2.0:1789760754.340056:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275776 00000100:00100000:2.0:1789760754.340130:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735524842403 time 735596440264 state 0x0 ref 1 00000100:00100000:2.0:1789760754.340266:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735596700680 time 735596440264 state 0x1 ref 1 00000100:00100000:2.0:1789760754.340300:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735596700680 time 735655602582 state 0x0 ref 1 00000100:00100000:2.0:1789760754.340315:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760754.340381:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20578304, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:2.0:1789760754.340393:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.340414:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760754.340420:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.340429:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.340447:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.422333:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20578304 -> 20582400 00010000:00010000:2.0:1789760754.422358:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422375:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422395:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422410:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.422416:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422426:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422451:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.422459:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b859b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.422587:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507275776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82266us (82950us total) trans 4294972344 rc 0/0 00000100:00100000:2.0:1789760754.422600:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5054 00000100:00100000:2.0:1789760754.422613:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735596700680 time 735655602582 state 0x2 ref 0 00000001:00080000:1.0:1789760754.423583:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972344 is committed 00000100:00100000:2.0:1789760754.428596:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.428760:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507275904 00000100:00100000:3.0:1789760754.428774:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507275904 00000100:00100000:3.0:1789760754.428849:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735596700680 time 735655602582 state 0x0 ref 1 00000100:00100000:3.0:1789760754.428868:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735655862998 time 735655602582 state 0x1 ref 1 00000100:00100000:3.0:1789760754.428905:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735655862998 time 735744208028 state 0x0 ref 1 00000100:00100000:3.0:1789760754.428920:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.428989:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20582400, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.429001:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.429022:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760754.429027:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.429035:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.429054:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.485135:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20582400 -> 20586496 00010000:00010000:0.0:1789760754.485165:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.485183:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.485188:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972345 is committed 00010000:00010000:0.0:1789760754.485193:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.485203:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760754.485209:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.485219:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.485248:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.485262:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.485477:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507275904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56549us (56885us total) trans 4294972345 rc 0/0 00000100:00100000:1.0:1789760754.485491:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5055 00000100:00100000:1.0:1789760754.485507:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735655862998 time 735744208028 state 0x2 ref 0 00000100:00100000:0.0:1789760754.491632:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760754.491743:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276032 00000100:00100000:1.0:1789760754.491756:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276032 00000100:00100000:1.0:1789760754.491829:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735655862998 time 735744208028 state 0x0 ref 1 00000100:00100000:1.0:1789760754.491848:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735744468444 time 735744208028 state 0x1 ref 1 00000100:00100000:1.0:1789760754.491888:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735744468444 time 735807191142 state 0x0 ref 1 00000100:00100000:1.0:1789760754.491902:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760754.491969:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20586496, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:1.0:1789760754.491980:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.492000:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760754.492006:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.492014:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.492032:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.527222:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972346 is committed 00002000:00010000:3.0:1789760754.527368:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:3.0:1789760754.527390:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527406:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527417:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527426:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.527432:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527444:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527472:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.527481:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b85a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.527609:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35701us (35981us total) trans 4294972346 rc 0/0 00000100:00100000:3.0:1789760754.527627:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5056 00000100:00100000:3.0:1789760754.527640:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735744468444 time 735807191142 state 0x2 ref 0 00000100:00100000:3.0:1789760754.532681:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760754.532894:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276160 00000100:00100000:2.0:1789760754.532910:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276160 00000100:00100000:2.0:1789760754.532987:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735744468444 time 735807191142 state 0x0 ref 1 00000100:00100000:2.0:1789760754.533021:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735807451558 time 735807191142 state 0x1 ref 1 00000100:00100000:2.0:1789760754.533882:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735807451558 time 735849177242 state 0x0 ref 1 00000100:00100000:2.0:1789760754.533906:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760754.533977:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20590592, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:2.0:1789760754.533988:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.534008:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760754.534012:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.534020:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.534037:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.588029:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760754.603296:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972347 is committed 00002000:00010000:2.0:1789760754.603416:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20590592 -> 20594688 00010000:00010000:2.0:1789760754.603439:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603461:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603471:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603479:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.603486:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603498:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603527:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.603535:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.603665:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69759us (70992us total) trans 4294972347 rc 0/0 00000100:00100000:2.0:1789760754.603677:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5057 00000100:00100000:2.0:1789760754.603743:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735807451558 time 735849177242 state 0x2 ref 0 00000100:00100000:2.0:1789760754.618429:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760754.618617:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276288 00000100:00100000:1.0:1789760754.618630:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276288 00000100:00100000:1.0:1789760754.618874:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735807451558 time 735849177242 state 0x0 ref 1 00000100:00100000:1.0:1789760754.618896:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735849437658 time 735849177242 state 0x1 ref 1 00000100:00100000:2.0:1789760754.620480:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735849437658 time 735935771053 state 0x0 ref 1 00000100:00100000:2.0:1789760754.620515:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507276288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760754.620589:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20594688, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:2.0:1789760754.620600:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.620620:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760754.620626:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.620634:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.620652:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.695536:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20594688 -> 20598784 00010000:00010000:2.0:1789760754.695566:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695582:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695592:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695601:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.695606:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695616:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695646:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.695654:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b85b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.695842:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507276288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75336us (77421us total) trans 4294972348 rc 0/0 00000100:00100000:2.0:1789760754.695855:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5058 00000100:00100000:2.0:1789760754.695868:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735849437658 time 735935771053 state 0x2 ref 0 00000001:00080000:0.0:1789760754.696252:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972348 is committed 00000100:00100000:0.0:1789760754.700269:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.700360:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276416 00000100:00100000:3.0:1789760754.700374:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276416 00000100:00100000:3.0:1789760754.700449:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735849437658 time 735935771053 state 0x0 ref 1 00000100:00100000:3.0:1789760754.700469:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735936031469 time 735935771053 state 0x1 ref 1 00000100:00100000:3.0:1789760754.700510:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735936031469 time 736015812346 state 0x0 ref 1 00000100:00100000:3.0:1789760754.700524:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.700594:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20598784, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.700606:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.700626:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.700632:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.700640:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.700658:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.731424:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972349 is committed 00002000:00010000:3.0:1789760754.731541:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20598784 -> 20602880 00010000:00010000:3.0:1789760754.731561:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731576:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731586:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731593:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.731599:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731610:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731633:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.731640:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.731939:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31409us (31676us total) trans 4294972349 rc 0/0 00000100:00100000:3.0:1789760754.731956:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5059 00000100:00100000:3.0:1789760754.731969:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735936031469 time 736015812346 state 0x2 ref 0 00000100:00100000:2.0:1789760754.735217:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.735429:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276544 00000100:00100000:3.0:1789760754.735444:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276544 00000100:00100000:3.0:1789760754.735510:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 735936031469 time 736015812346 state 0x0 ref 1 00000100:00100000:3.0:1789760754.735528:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736016072762 time 736015812346 state 0x1 ref 1 00000100:00100000:3.0:1789760754.735607:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736016072762 time 736050909658 state 0x0 ref 1 00000100:00100000:3.0:1789760754.735622:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.735683:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20602880, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.735731:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.735751:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.735756:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.735763:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.735779:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.760602:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972350 is committed 00002000:00010000:2.0:1789760754.761227:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20602880 -> 20606976 00010000:00010000:2.0:1789760754.761488:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761507:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761516:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761524:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.761530:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761541:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761569:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.761579:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.762917:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27282us (27697us total) trans 4294972350 rc 0/0 00000100:00100000:2.0:1789760754.762933:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5060 00000100:00100000:2.0:1789760754.762946:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736016072762 time 736050909658 state 0x2 ref 0 00000100:00100000:2.0:1789760754.766440:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.766570:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276672 00000100:00100000:3.0:1789760754.766585:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276672 00000100:00100000:3.0:1789760754.766660:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736016072762 time 736050909658 state 0x0 ref 1 00000100:00100000:3.0:1789760754.766680:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736051170074 time 736050909658 state 0x1 ref 1 00000100:00100000:3.0:1789760754.766928:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736051170074 time 736082229479 state 0x0 ref 1 00000100:00100000:3.0:1789760754.766943:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507276672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.767006:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20606976, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.767016:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.767035:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.767040:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.767047:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.767064:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.807966:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00002000:00010000:0.0:1789760754.808105:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20606976 -> 20611072 00010000:00010000:0.0:1789760754.808125:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808257:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808269:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808278:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760754.808284:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808295:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808322:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.808331:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.808470:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507276672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41521us (42038us total) trans 4294972351 rc 0/0 00000100:00100000:0.0:1789760754.808482:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5061 00000100:00100000:0.0:1789760754.808495:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736051170074 time 736082229479 state 0x2 ref 0 00000100:00100000:0.0:1789760754.813487:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.813582:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276800 00000100:00100000:3.0:1789760754.813595:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276800 00000100:00100000:3.0:1789760754.813667:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736051170074 time 736082229479 state 0x0 ref 1 00000100:00100000:3.0:1789760754.813686:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736082489895 time 736082229479 state 0x1 ref 1 00000100:00100000:3.0:1789760754.813865:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736082489895 time 736129167288 state 0x0 ref 1 00000100:00100000:3.0:1789760754.813881:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507276800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.813950:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20611072, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.813961:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813981:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.813986:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.813993:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.814011:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.845187:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20611072 -> 20615168 00010000:00010000:2.0:1789760754.845214:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845229:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845240:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845248:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760754.845254:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845264:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845293:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.845300:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b85d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.845431:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507276800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31545us (31951us total) trans 4294972352 rc 0/0 00000100:00100000:2.0:1789760754.845443:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5062 00000100:00100000:2.0:1789760754.845457:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736082489895 time 736129167288 state 0x2 ref 0 00000001:00080000:1.0:1789760754.847413:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00000100:00100000:2.0:1789760754.849078:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.849261:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507276928 00000100:00100000:3.0:1789760754.849275:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507276928 00000100:00100000:3.0:1789760754.849355:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736082489895 time 736129167288 state 0x0 ref 1 00000100:00100000:3.0:1789760754.849375:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736129427704 time 736129167288 state 0x1 ref 1 00000100:00100000:3.0:1789760754.849415:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736129427704 time 736164717311 state 0x0 ref 1 00000100:00100000:3.0:1789760754.849428:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.849500:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20615168, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.849511:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.849532:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.849538:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.849546:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.849565:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.887746:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972353 is committed 00002000:00010000:3.0:1789760754.888073:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20615168 -> 20619264 00010000:00010000:3.0:1789760754.888096:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888113:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888123:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888131:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.888138:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888148:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888173:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.888182:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.888432:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507276928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38998us (39359us total) trans 4294972353 rc 0/0 00000100:00100000:3.0:1789760754.888446:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5063 00000100:00100000:3.0:1789760754.888459:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736129427704 time 736164717311 state 0x2 ref 0 00000100:00100000:3.0:1789760754.892091:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760754.892337:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277056 00000100:00100000:0.0:1789760754.892352:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277056 00000100:00100000:0.0:1789760754.892430:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736129427704 time 736164717311 state 0x0 ref 1 00000100:00100000:0.0:1789760754.892450:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736164977727 time 736164717311 state 0x1 ref 1 00000100:00100000:0.0:1789760754.892541:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736164977727 time 736207843879 state 0x0 ref 1 00000100:00100000:0.0:1789760754.892554:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760754.892616:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20619264, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:0.0:1789760754.892626:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.892646:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760754.892651:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.892660:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.892676:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.908114:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538280064:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760754.908209:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760754.908448:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280064 00000100:00100000:2.0:1789760754.908459:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280064 00000100:00100000:2.0:1789760754.908515:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789760754.908527:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538280064:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760754.908556:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3837788160 avail 3830448128 00000100:00100000:0.0:1789760754.908820:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760754.908833:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538280192:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760754.908866:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760754.908902:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280192 00000100:00100000:0.0:1789760754.908906:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280192 00000100:00100000:0.0:1789760754.908934:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:0.0:1789760754.908955:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538280192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760754.908967:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3837788160 avail 3830448128 00000100:00100000:0.0:1789760754.908999:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538280192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760754.909007:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:0.0:1789760754.909060:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538280192:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760754.911363:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538280064:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2825us (3140us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760754.911379:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789760754.911540:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538280064:0@lo:13:osp-pre-0-1.0 00002000:00010000:3.0:1789760754.933876:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:3.0:1789760754.933905:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933922:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933943:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933952:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.933958:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933968:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933994:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.934009:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b85e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.934156:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507277056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41596us (42072us total) trans 4294972354 rc 0/0 00000100:00100000:3.0:1789760754.934168:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5064 00000100:00100000:3.0:1789760754.934181:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736164977727 time 736207843879 state 0x2 ref 0 00000001:00080000:0.0:1789760754.934780:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972354 is committed 00000100:00100000:1.0:1789760754.939262:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.939479:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277184 00000100:00100000:3.0:1789760754.939567:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277184 00000100:00100000:3.0:1789760754.939646:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736164977727 time 736207843879 state 0x0 ref 1 00000100:00100000:3.0:1789760754.939666:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736208104295 time 736207843879 state 0x1 ref 1 00000100:00100000:3.0:1789760754.940256:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736208104295 time 736255545438 state 0x0 ref 1 00000100:00100000:3.0:1789760754.940275:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.940342:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20623360, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.940354:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.940374:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.940379:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.940388:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.940410:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.988775:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972355 is committed 00002000:00010000:3.0:1789760754.988804:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20623360 -> 20627456 00010000:00010000:3.0:1789760754.988829:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988846:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988858:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988867:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760754.988873:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988884:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988913:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.988933:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.989072:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48794us (49831us total) trans 4294972355 rc 0/0 00000100:00100000:3.0:1789760754.989094:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5065 00000100:00100000:3.0:1789760754.989109:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736208104295 time 736255545438 state 0x2 ref 0 00000100:00100000:1.0:1789760754.995334:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760754.995464:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277312 00000100:00100000:3.0:1789760754.995477:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277312 00000100:00100000:3.0:1789760754.995543:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736208104295 time 736255545438 state 0x0 ref 1 00000100:00100000:3.0:1789760754.995615:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736255805854 time 736255545438 state 0x1 ref 1 00000100:00100000:3.0:1789760754.995681:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736255805854 time 736310983025 state 0x0 ref 1 00000100:00100000:3.0:1789760754.995718:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760754.995792:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20627456, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa007 00010000:00010000:3.0:1789760754.995803:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.995824:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760754.995830:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.995838:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.995855:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.046044:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972356 is committed 00002000:00010000:0.0:1789760755.046619:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20627456 -> 20631552 00002000:00010000:0.0:1789760755.046627:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:0.0:1789760755.046629:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:0.0:1789760755.046646:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046663:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046673:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046681:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.046737:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046753:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046780:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046790:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b85ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.046934:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51201us (51599us total) trans 4294972356 rc 0/0 00000100:00100000:0.0:1789760755.046946:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5066 00000100:00100000:0.0:1789760755.046958:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736255805854 time 736310983025 state 0x2 ref 0 00000100:00100000:2.0:1789760755.059109:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760755.064198:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277440 00000100:00100000:3.0:1789760755.064222:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277440 00000100:00100000:3.0:1789760755.064399:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736255805854 time 736310983025 state 0x0 ref 1 00000100:00100000:3.0:1789760755.064420:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736311243441 time 736310983025 state 0x1 ref 1 00000100:00100000:3.0:1789760755.064462:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736311243441 time 736379764613 state 0x0 ref 1 00000100:00100000:3.0:1789760755.064483:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760755.064557:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20631552, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:3.0:1789760755.064583:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.064604:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760755.064610:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.064619:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.064639:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760755.065404:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760755.107070:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20631552 -> 20635648 00010000:00010000:1.0:1789760755.107099:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107114:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107124:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107203:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.107211:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107222:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107320:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.107330:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b85f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.107509:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507277440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43027us (48409us total) trans 4294972357 rc 0/0 00000100:00100000:1.0:1789760755.107523:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5067 00000100:00100000:1.0:1789760755.107535:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736311243441 time 736379764613 state 0x2 ref 0 00000001:00080000:1.0:1789760755.107590:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972357 is committed 00000100:00100000:1.0:1789760755.117147:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760755.117280:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277568 00000100:00100000:3.0:1789760755.117295:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277568 00000100:00100000:3.0:1789760755.117372:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736311243441 time 736379764613 state 0x0 ref 1 00000100:00100000:3.0:1789760755.117391:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736380025029 time 736379764613 state 0x1 ref 1 00000100:00100000:3.0:1789760755.117426:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736380025029 time 736432728532 state 0x0 ref 1 00000100:00100000:3.0:1789760755.117440:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760755.117507:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20635648, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:3.0:1789760755.117518:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.117537:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760755.117543:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.117551:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.117568:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.171327:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972358 is committed 00002000:00010000:0.0:1789760755.171469:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20635648 -> 20639744 00010000:00010000:0.0:1789760755.171489:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171505:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171517:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171525:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.171532:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171548:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171573:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.171581:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b85fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.171757:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54311us (54625us total) trans 4294972358 rc 0/0 00000100:00100000:0.0:1789760755.171769:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5068 00000100:00100000:0.0:1789760755.171781:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736380025029 time 736432728532 state 0x2 ref 0 00000100:00100000:3.0:1789760755.178454:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.178566:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277696 00000100:00100000:0.0:1789760755.178578:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277696 00000100:00100000:0.0:1789760755.178648:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736380025029 time 736432728532 state 0x0 ref 1 00000100:00100000:0.0:1789760755.178666:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736432988948 time 736432728532 state 0x1 ref 1 00000100:00100000:1.0:1789760755.178819:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736432988948 time 736494118219 state 0x0 ref 1 00000100:00100000:1.0:1789760755.178838:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507277696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760755.179121:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20639744, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:1.0:1789760755.179134:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.179160:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760755.179166:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.179173:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.179192:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.210851:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972359 is committed 00002000:00010000:0.0:1789760755.210993:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20639744 -> 20643840 00010000:00010000:0.0:1789760755.211013:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211030:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211046:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211054:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.211060:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211070:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211095:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.211103:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8604 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.211225:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507277696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32385us (32885us total) trans 4294972359 rc 0/0 00000100:00100000:0.0:1789760755.211237:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5069 00000100:00100000:0.0:1789760755.211250:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736432988948 time 736494118219 state 0x2 ref 0 00000100:00100000:2.0:1789760755.214465:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760755.214922:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277824 00000100:00100000:1.0:1789760755.214938:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277824 00000100:00100000:1.0:1789760755.215019:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736432988948 time 736494118219 state 0x0 ref 1 00000100:00100000:1.0:1789760755.215040:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736494378635 time 736494118219 state 0x1 ref 1 00000100:00100000:1.0:1789760755.215077:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736494378635 time 736530379664 state 0x0 ref 1 00000100:00100000:1.0:1789760755.215092:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760755.215172:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20643840, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:1.0:1789760755.215183:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.215204:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760755.215210:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.215219:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.215242:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.253713:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972360 is committed 00002000:00010000:1.0:1789760755.253933:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20643840 -> 20647936 00010000:00010000:1.0:1789760755.253954:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.253970:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.253981:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.253991:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.253997:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.254079:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.254107:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.254116:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b860b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.254325:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507277824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39227us (39867us total) trans 4294972360 rc 0/0 00000100:00100000:1.0:1789760755.254338:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5070 00000100:00100000:1.0:1789760755.254350:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736494378635 time 736530379664 state 0x2 ref 0 00000100:00100000:2.0:1789760755.258524:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.258629:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507277952 00000100:00100000:0.0:1789760755.258642:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507277952 00000100:00100000:0.0:1789760755.258757:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736494378635 time 736530379664 state 0x0 ref 1 00000100:00100000:0.0:1789760755.258775:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736530640080 time 736530379664 state 0x1 ref 1 00000100:00100000:0.0:1789760755.258813:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736530640080 time 736574116297 state 0x0 ref 1 00000100:00100000:0.0:1789760755.258828:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507277952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.258898:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20647936, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.258909:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.258930:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760755.258935:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.258943:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.258962:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.305432:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00002000:00010000:0.0:1789760755.305558:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20647936 -> 20652032 00010000:00010000:0.0:1789760755.305578:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305594:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305604:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305612:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.305618:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305628:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305654:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.305663:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8612 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.305836:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507277952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47002us (47318us total) trans 4294972361 rc 0/0 00000100:00100000:0.0:1789760755.305850:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5071 00000100:00100000:0.0:1789760755.305863:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736530640080 time 736574116297 state 0x2 ref 0 00000100:00100000:3.0:1789760755.308603:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.309129:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278080 00000100:00100000:0.0:1789760755.309213:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278080 00000100:00100000:0.0:1789760755.309296:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736530640080 time 736574116297 state 0x0 ref 1 00000100:00100000:0.0:1789760755.309317:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736574376713 time 736574116297 state 0x1 ref 1 00000100:00100000:0.0:1789760755.309353:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736574376713 time 736624655340 state 0x0 ref 1 00000100:00100000:0.0:1789760755.309368:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507278080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.309434:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20652032, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.309447:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.309469:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760755.309476:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.309485:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.309503:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.351105:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972362 is committed 00002000:00010000:1.0:1789760755.351259:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20652032 -> 20656128 00010000:00010000:1.0:1789760755.351279:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351299:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351309:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351317:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.351324:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351335:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351361:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.351374:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8619 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.351531:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507278080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42158us (42936us total) trans 4294972362 rc 0/0 00000100:00100000:1.0:1789760755.351544:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5072 00000100:00100000:1.0:1789760755.351557:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736574376713 time 736624655340 state 0x2 ref 0 00000100:00100000:3.0:1789760755.355374:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760755.355450:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278208 00000100:00100000:2.0:1789760755.355463:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278208 00000100:00100000:2.0:1789760755.355542:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736574376713 time 736624655340 state 0x0 ref 1 00000100:00100000:2.0:1789760755.355561:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736624915756 time 736624655340 state 0x1 ref 1 00000100:00100000:2.0:1789760755.355594:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736624915756 time 736670896736 state 0x0 ref 1 00000100:00100000:2.0:1789760755.355608:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507278208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760755.355673:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20656128, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:2.0:1789760755.355714:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.355736:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760755.355741:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.355749:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.355765:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.394032:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20656128 -> 20660224 00010000:00010000:1.0:1789760755.394061:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394077:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394088:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394095:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.394101:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394111:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394151:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.394160:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8620 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.394290:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507278208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38677us (38926us total) trans 4294972363 rc 0/0 00000100:00100000:1.0:1789760755.394303:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5073 00000100:00100000:1.0:1789760755.394316:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736624915756 time 736670896736 state 0x2 ref 0 00000001:00080000:3.0:1789760755.398331:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972363 is committed 00000100:00100000:1.0:1789760755.399242:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.399359:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278336 00000100:00100000:0.0:1789760755.399373:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278336 00000100:00100000:0.0:1789760755.399445:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736624915756 time 736670896736 state 0x0 ref 1 00000100:00100000:0.0:1789760755.399465:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736671157152 time 736670896736 state 0x1 ref 1 00000100:00100000:0.0:1789760755.399546:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736671157152 time 736714848093 state 0x0 ref 1 00000100:00100000:0.0:1789760755.399562:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507278336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.399630:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20660224, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.399642:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.399662:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760755.399668:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.399676:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.399870:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.420019:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538280320:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760755.420148:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760755.420453:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280320 00000100:00100000:1.0:1789760755.420467:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280320 00000100:00100000:1.0:1789760755.420539:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1789760755.420550:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538280320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760755.420602:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760755.420673:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538280320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 124us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760755.420682:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1789760755.420884:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538280320:0@lo:41:osp-pre-1-0.0 00002000:00010000:0.0:1789760755.443071:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20660224 -> 20664320 00010000:00010000:0.0:1789760755.443098:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443115:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443124:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443198:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.443205:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443215:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443240:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.443254:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8627 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.443374:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507278336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43807us (44133us total) trans 4294972364 rc 0/0 00000100:00100000:0.0:1789760755.443387:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5074 00000100:00100000:0.0:1789760755.443399:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736671157152 time 736714848093 state 0x2 ref 0 00000100:00100000:3.0:1789760755.446075:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.446392:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278464 00000100:00100000:0.0:1789760755.446407:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278464 00000100:00100000:0.0:1789760755.446482:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736671157152 time 736714848093 state 0x0 ref 1 00000100:00100000:0.0:1789760755.446503:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736715108509 time 736714848093 state 0x1 ref 1 00000100:00100000:0.0:1789760755.446531:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736715108509 time 736761833943 state 0x0 ref 1 00000100:00100000:0.0:1789760755.446545:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507278464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.446614:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20664320, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.446624:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.446644:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760755.446649:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.446658:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.446679:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.447522:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972364 is committed 00000001:00080000:2.0:1789760755.482397:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972365 is committed 00002000:00010000:0.0:1789760755.482548:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:0.0:1789760755.482570:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482586:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482595:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482603:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.482608:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482618:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482644:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.482654:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b862e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.482939:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507278464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36389us (36872us total) trans 4294972365 rc 0/0 00000100:00100000:0.0:1789760755.482952:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5075 00000100:00100000:0.0:1789760755.482965:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736715108509 time 736761833943 state 0x2 ref 0 00000100:00100000:3.0:1789760755.486967:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760755.490990:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278592 00000100:00100000:1.0:1789760755.491011:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278592 00000100:00100000:1.0:1789760755.491105:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736715108509 time 736761833943 state 0x0 ref 1 00000100:00100000:1.0:1789760755.491273:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736762094359 time 736761833943 state 0x1 ref 1 00000100:00100000:1.0:1789760755.491323:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736762094359 time 736806624748 state 0x0 ref 1 00000100:00100000:1.0:1789760755.491339:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507278592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760755.491408:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20668416, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:1.0:1789760755.491419:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.491439:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760755.491444:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.491452:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.491469:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.537855:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972366 is committed 00002000:00010000:2.0:1789760755.538013:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20668416 -> 20672512 00010000:00010000:2.0:1789760755.538036:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538051:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538062:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538069:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760755.538075:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538085:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538122:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.538131:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8635 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.538262:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507278592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46920us (51302us total) trans 4294972366 rc 0/0 00000100:00100000:2.0:1789760755.538276:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5076 00000100:00100000:2.0:1789760755.538288:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736762094359 time 736806624748 state 0x2 ref 0 00000100:00100000:3.0:1789760755.541615:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.541763:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278720 00000100:00100000:0.0:1789760755.541775:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278720 00000100:00100000:0.0:1789760755.541850:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736762094359 time 736806624748 state 0x0 ref 1 00000100:00100000:0.0:1789760755.541870:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736806885164 time 736806624748 state 0x1 ref 1 00000100:00100000:0.0:1789760755.541907:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736806885164 time 736857209692 state 0x0 ref 1 00000100:00100000:0.0:1789760755.541922:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507278720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.541988:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20672512, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.541999:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.542018:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760755.542023:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.542030:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.542046:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.594521:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00002000:00010000:3.0:1789760755.594663:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20672512 -> 20676608 00010000:00010000:3.0:1789760755.594712:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595369:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595381:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595388:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760755.595396:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595406:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595437:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.595446:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b863c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.595965:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507278720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54032us (54353us total) trans 4294972367 rc 0/0 00000100:00100000:3.0:1789760755.595980:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5077 00000100:00100000:3.0:1789760755.596003:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736806885164 time 736857209692 state 0x2 ref 0 00000100:00100000:0.0:1789760755.599122:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760755.599449:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278848 00000100:00100000:1.0:1789760755.599463:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278848 00000100:00100000:1.0:1789760755.599545:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736806885164 time 736857209692 state 0x0 ref 1 00000100:00100000:1.0:1789760755.599561:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736857470108 time 736857209692 state 0x1 ref 1 00000100:00100000:1.0:1789760755.599627:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736857470108 time 736914928779 state 0x0 ref 1 00000100:00100000:1.0:1789760755.599642:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507278848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760755.599904:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20676608, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:1.0:1789760755.599916:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.599937:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760755.599943:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.599951:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.599970:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.612081:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760755.632303:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20676608 -> 20680704 00010000:00010000:1.0:1789760755.632406:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632422:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632432:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632440:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.632445:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632455:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632487:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.632495:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8643 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.632620:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507278848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32973us (33506us total) trans 4294972368 rc 0/0 00000100:00100000:1.0:1789760755.632631:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5078 00000100:00100000:1.0:1789760755.632643:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736857470108 time 736914928779 state 0x2 ref 0 00000100:00100000:1.0:1789760755.636988:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760755.637071:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507278976 00000100:00100000:3.0:1789760755.637086:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507278976 00000100:00100000:3.0:1789760755.637225:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736857470108 time 736914928779 state 0x0 ref 1 00000100:00100000:3.0:1789760755.637257:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736915189195 time 736914928779 state 0x1 ref 1 00000100:00100000:3.0:1789760755.637294:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736915189195 time 736952596901 state 0x0 ref 1 00000100:00100000:3.0:1789760755.637310:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507278976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760755.637375:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20680704, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:3.0:1789760755.637387:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.637420:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760755.637426:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.637434:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.637452:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.637554:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972368 is committed 00002000:00010000:2.0:1789760755.683354:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20680704 -> 20684800 00010000:00010000:2.0:1789760755.683380:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683396:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683406:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683414:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760755.683420:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683431:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683457:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.683467:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b864a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.683601:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507278976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46287us (46621us total) trans 4294972369 rc 0/0 00000100:00100000:2.0:1789760755.683614:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5079 00000100:00100000:2.0:1789760755.683627:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736915189195 time 736952596901 state 0x2 ref 0 00000100:00100000:1.0:1789760755.686593:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760755.686678:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279104 00000100:00100000:3.0:1789760755.686715:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279104 00000100:00100000:3.0:1789760755.686793:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736915189195 time 736952596901 state 0x0 ref 1 00000100:00100000:3.0:1789760755.686811:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736952857317 time 736952596901 state 0x1 ref 1 00000100:00100000:3.0:1789760755.686845:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736952857317 time 737002148327 state 0x0 ref 1 00000100:00100000:3.0:1789760755.686858:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507279104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760755.686918:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20684800, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:3.0:1789760755.686928:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.686946:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760755.686951:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.686957:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.686975:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.687941:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972369 is committed 00000001:00080000:1.0:1789760755.716736:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972370 is committed 00002000:00010000:0.0:1789760755.716765:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20684800 -> 20688896 00010000:00010000:0.0:1789760755.716786:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716802:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716812:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716821:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760755.716828:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716839:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716864:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.716874:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8651 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.717004:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30139us (30418us total) trans 4294972370 rc 0/0 00000100:00100000:0.0:1789760755.717017:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5080 00000100:00100000:0.0:1789760755.717028:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736952857317 time 737002148327 state 0x2 ref 0 00000100:00100000:1.0:1789760755.720070:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.722238:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279232 00000100:00100000:0.0:1789760755.722258:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279232 00000100:00100000:0.0:1789760755.722358:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 736952857317 time 737002148327 state 0x0 ref 1 00000100:00100000:0.0:1789760755.722377:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737002408743 time 737002148327 state 0x1 ref 1 00000100:00100000:0.0:1789760755.722404:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737002408743 time 737037707380 state 0x0 ref 1 00000100:00100000:0.0:1789760755.722419:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.722487:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20688896, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.722498:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.722518:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760755.722523:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.722531:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.722548:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.762943:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20688896 -> 20692992 00010000:00010000:1.0:1789760755.762971:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.762988:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.762999:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.763006:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760755.763013:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.763024:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.763048:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.763056:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8658 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.763188:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507279232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40765us (43126us total) trans 4294972371 rc 0/0 00000100:00100000:1.0:1789760755.763200:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5081 00000100:00100000:1.0:1789760755.763218:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737002408743 time 737037707380 state 0x2 ref 0 00000001:00080000:0.0:1789760755.763419:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972371 is committed 00000100:00100000:1.0:1789760755.769642:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760755.769883:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279360 00000100:00100000:2.0:1789760755.769897:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279360 00000100:00100000:2.0:1789760755.769973:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737002408743 time 737037707380 state 0x0 ref 1 00000100:00100000:2.0:1789760755.769991:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737037967796 time 737037707380 state 0x1 ref 1 00000100:00100000:2.0:1789760755.770024:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737037967796 time 737085326746 state 0x0 ref 1 00000100:00100000:2.0:1789760755.770040:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760755.770102:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20692992, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:2.0:1789760755.770125:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.770146:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760755.770151:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.770159:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.770175:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.810889:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972372 is committed 00002000:00010000:2.0:1789760755.811008:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20692992 -> 20697088 00010000:00010000:2.0:1789760755.811028:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811050:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811060:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811069:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760755.811074:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811084:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811109:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.811118:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b865f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.811302:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41258us (41667us total) trans 4294972372 rc 0/0 00000100:00100000:2.0:1789760755.811315:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5082 00000100:00100000:2.0:1789760755.811327:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737037967796 time 737085326746 state 0x2 ref 0 00000100:00100000:2.0:1789760755.814105:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760755.814227:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279488 00000100:00100000:0.0:1789760755.814240:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279488 00000100:00100000:0.0:1789760755.814310:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737037967796 time 737085326746 state 0x0 ref 1 00000100:00100000:0.0:1789760755.814329:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737085587162 time 737085326746 state 0x1 ref 1 00000100:00100000:0.0:1789760755.814363:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737085587162 time 737129665772 state 0x0 ref 1 00000100:00100000:0.0:1789760755.814381:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760755.814446:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20697088, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:0.0:1789760755.814457:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.814476:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760755.814481:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.814487:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.814504:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.868513:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972373 is committed 00002000:00010000:3.0:1789760755.869083:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20697088 -> 20701184 00010000:00010000:3.0:1789760755.869103:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869119:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869139:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869148:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760755.869154:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869164:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869190:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.869199:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8666 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.869338:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54955us (55239us total) trans 4294972373 rc 0/0 00000100:00100000:3.0:1789760755.869350:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5083 00000100:00100000:3.0:1789760755.869363:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737085587162 time 737129665772 state 0x2 ref 0 00000100:00100000:1.0:1789760755.874190:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760755.874347:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279616 00000100:00100000:2.0:1789760755.874362:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279616 00000100:00100000:2.0:1789760755.874439:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737085587162 time 737129665772 state 0x0 ref 1 00000100:00100000:2.0:1789760755.874459:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737129926188 time 737129665772 state 0x1 ref 1 00000100:00100000:2.0:1789760755.874498:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737129926188 time 737189800528 state 0x0 ref 1 00000100:00100000:2.0:1789760755.874512:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760755.874578:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20701184, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:2.0:1789760755.874589:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.874610:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760755.874615:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.874622:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.874639:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.933300:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1051c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538280448:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760755.933399:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760755.933496:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280448 00000100:00100000:3.0:1789760755.933505:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280448 00000100:00100000:3.0:1789760755.933561:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789760755.933572:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538280448:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760755.933619:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760755.933676:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538280448:12345-0@lo:41:osp-pre-0-1.0 Request processed in 105us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.933713:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789760755.933767:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1051c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538280448:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760755.952669:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972374 is committed 00002000:00010000:2.0:1789760755.952889:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20701184 -> 20705280 00010000:00010000:2.0:1789760755.952910:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.952926:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.952942:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.952951:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760755.952957:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.952968:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.952992:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.953000:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b866d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.953129:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78612us (78946us total) trans 4294972374 rc 0/0 00000100:00100000:2.0:1789760755.953141:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5084 00000100:00100000:2.0:1789760755.953153:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737129926188 time 737189800528 state 0x2 ref 0 00000100:00100000:1.0:1789760755.956665:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760755.956757:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279744 00000100:00100000:2.0:1789760755.956771:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279744 00000100:00100000:2.0:1789760755.956848:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737129926188 time 737189800528 state 0x0 ref 1 00000100:00100000:2.0:1789760755.956868:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737190060944 time 737189800528 state 0x1 ref 1 00000100:00100000:1.0:1789760755.957259:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737190060944 time 737272492807 state 0x0 ref 1 00000100:00100000:1.0:1789760755.957280:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507279744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760755.957349:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20705280, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:1.0:1789760755.957360:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.957380:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760755.957385:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.957393:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.957410:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.006653:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972375 is committed 00002000:00010000:0.0:1789760756.006880:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20705280 -> 20709376 00010000:00010000:0.0:1789760756.006901:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006915:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006925:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006933:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760756.006939:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006949:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006975:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.006984:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8674 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.007121:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507279744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49837us (50462us total) trans 4294972375 rc 0/0 00000100:00100000:0.0:1789760756.007142:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5085 00000100:00100000:0.0:1789760756.007154:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737190060944 time 737272492807 state 0x2 ref 0 00000100:00100000:1.0:1789760756.010662:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760756.010897:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507279872 00000100:00100000:3.0:1789760756.010912:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507279872 00000100:00100000:3.0:1789760756.010987:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737190060944 time 737272492807 state 0x0 ref 1 00000100:00100000:3.0:1789760756.011008:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737272753223 time 737272492807 state 0x1 ref 1 00000100:00100000:3.0:1789760756.011366:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737272753223 time 737326666518 state 0x0 ref 1 00000100:00100000:3.0:1789760756.011384:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760756.011466:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20709376, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xa007 00010000:00010000:3.0:1789760756.011478:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.011500:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760756.011506:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.011515:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.011533:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.058355:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00002000:00010000:0.0:1789760756.058544:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20709376 -> 20713472 00002000:00010000:0.0:1789760756.058550:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:0.0:1789760756.058552:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:0.0:1789760756.058569:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058585:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058596:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058604:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760756.058610:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058620:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058644:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.058652:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b867b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.058826:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507279872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47438us (48172us total) trans 4294972376 rc 0/0 00000100:00100000:0.0:1789760756.058838:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5086 00000100:00100000:0.0:1789760756.058850:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737272753223 time 737326666518 state 0x2 ref 0 00000100:00100000:1.0:1789760756.063155:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760756.065003:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280000 00000100:00100000:2.0:1789760756.065022:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280000 00000100:00100000:2.0:1789760756.065108:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737272753223 time 737326666518 state 0x0 ref 1 00000100:00100000:2.0:1789760756.065129:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737326926934 time 737326666518 state 0x1 ref 1 00000100:00100000:2.0:1789760756.065380:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737326926934 time 737380680908 state 0x0 ref 1 00000100:00100000:2.0:1789760756.065399:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760756.065467:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20713472, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760756.065478:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.065499:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760756.065620:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.065631:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.065653:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.106400:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00002000:00010000:3.0:1789760756.106606:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20713472 -> 20717568 00010000:00010000:3.0:1789760756.106626:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106642:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106653:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106661:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760756.106667:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106675:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106797:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106806:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8682 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.106967:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41567us (43822us total) trans 4294972377 rc 0/0 00000100:00100000:3.0:1789760756.106979:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5087 00000100:00100000:3.0:1789760756.106991:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737326926934 time 737380680908 state 0x2 ref 0 00000100:00100000:2.0:1789760756.113992:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760756.114209:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280128 00000100:00100000:0.0:1789760756.114223:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280128 00000100:00100000:0.0:1789760756.114296:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737326926934 time 737380680908 state 0x0 ref 1 00000100:00100000:0.0:1789760756.114316:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737380941324 time 737380680908 state 0x1 ref 1 00000100:00100000:0.0:1789760756.114353:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737380941324 time 737429655827 state 0x0 ref 1 00000100:00100000:0.0:1789760756.114368:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760756.114433:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20717568, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:0.0:1789760756.114444:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.114468:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760756.114475:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.114483:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.114501:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.173759:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972378 is committed 00002000:00010000:0.0:1789760756.173954:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20717568 -> 20721664 00010000:00010000:0.0:1789760756.173972:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.173986:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.173995:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.174003:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760756.174008:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.174018:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.174044:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.174063:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8689 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.174187:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59815us (60207us total) trans 4294972378 rc 0/0 00000100:00100000:0.0:1789760756.174200:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5088 00000100:00100000:0.0:1789760756.174212:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737380941324 time 737429655827 state 0x2 ref 0 00000100:00100000:1.0:1789760756.179010:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760756.179367:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280256 00000100:00100000:3.0:1789760756.179380:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280256 00000100:00100000:0.0:1789760756.180064:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737380941324 time 737429655827 state 0x0 ref 1 00000100:00100000:0.0:1789760756.180085:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737429916243 time 737429655827 state 0x1 ref 1 00000100:00100000:0.0:1789760756.180122:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737429916243 time 737495425580 state 0x0 ref 1 00000100:00100000:0.0:1789760756.180138:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507280256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760756.180207:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20721664, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:0.0:1789760756.180218:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.180236:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760756.180241:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.180249:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.180267:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.264298:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20721664 -> 20725760 00010000:00010000:1.0:1789760756.264330:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.264330:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972379 is committed 00010000:00010000:1.0:1789760756.264349:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.264360:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.264369:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.264375:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.264391:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.264418:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.264426:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8690 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.264554:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507280256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 84413us (85553us total) trans 4294972379 rc 0/0 00000100:00100000:1.0:1789760756.264568:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5089 00000100:00100000:1.0:1789760756.264580:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737429916243 time 737495425580 state 0x2 ref 0 00000100:00100000:3.0:1789760756.274244:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760756.274454:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280384 00000100:00100000:2.0:1789760756.274466:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280384 00000100:00100000:2.0:1789760756.274547:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737429916243 time 737495425580 state 0x0 ref 1 00000100:00100000:2.0:1789760756.274567:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737495685996 time 737495425580 state 0x1 ref 1 00000100:00100000:2.0:1789760756.274655:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737495685996 time 737589957385 state 0x0 ref 1 00000100:00100000:2.0:1789760756.274671:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760756.274937:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20725760, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760756.274950:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.274971:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760756.274976:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.274984:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.275001:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.315063:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972380 is committed 00002000:00010000:3.0:1789760756.315197:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20725760 -> 20729856 00010000:00010000:3.0:1789760756.315218:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315232:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315243:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315251:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760756.315256:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315267:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315291:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.315299:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8697 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.315503:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507280384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40827us (41265us total) trans 4294972380 rc 0/0 00000100:00100000:3.0:1789760756.315516:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5090 00000100:00100000:3.0:1789760756.315528:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737495685996 time 737589957385 state 0x2 ref 0 00000100:00100000:1.0:1789760756.318460:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760756.318721:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280512 00000100:00100000:0.0:1789760756.318735:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280512 00000100:00100000:0.0:1789760756.318906:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737495685996 time 737589957385 state 0x0 ref 1 00000100:00100000:0.0:1789760756.318921:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737590217801 time 737589957385 state 0x1 ref 1 00000100:00100000:0.0:1789760756.318957:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737590217801 time 737634259526 state 0x0 ref 1 00000100:00100000:0.0:1789760756.318973:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507280512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760756.319038:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20729856, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:0.0:1789760756.319049:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.319069:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760756.319074:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.319081:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.319099:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.364096:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20729856 -> 20733952 00010000:00010000:1.0:1789760756.364136:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364154:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364165:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364179:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.364184:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364194:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364220:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.364229:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b869e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.364363:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507280512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45385us (45910us total) trans 4294972381 rc 0/0 00000100:00100000:1.0:1789760756.364376:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5091 00000100:00100000:1.0:1789760756.364390:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737590217801 time 737634259526 state 0x2 ref 0 00000001:00080000:3.0:1789760756.364764:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972381 is committed 00000100:00100000:2.0:1789760756.367938:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.368036:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280640 00000100:00100000:1.0:1789760756.368051:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280640 00000100:00100000:1.0:1789760756.368128:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737590217801 time 737634259526 state 0x0 ref 1 00000100:00100000:1.0:1789760756.368149:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737634519942 time 737634259526 state 0x1 ref 1 00000100:00100000:1.0:1789760756.368189:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737634519942 time 737683491699 state 0x0 ref 1 00000100:00100000:1.0:1789760756.368209:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507280640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.368275:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20733952, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.368285:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.368305:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760756.368310:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.368319:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.368338:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.412049:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972382 is committed 00002000:00010000:1.0:1789760756.412444:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20733952 -> 20738048 00010000:00010000:1.0:1789760756.412466:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412482:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412494:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412654:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.412661:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412673:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412955:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.412977:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.413183:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507280640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44972us (45256us total) trans 4294972382 rc 0/0 00000100:00100000:1.0:1789760756.413196:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5092 00000100:00100000:1.0:1789760756.413208:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737634519942 time 737683491699 state 0x2 ref 0 00000100:00100000:2.0:1789760756.416436:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760756.416841:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280768 00000100:00100000:3.0:1789760756.416856:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280768 00000100:00100000:3.0:1789760756.416930:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737634519942 time 737683491699 state 0x0 ref 1 00000100:00100000:3.0:1789760756.416949:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737683752115 time 737683491699 state 0x1 ref 1 00000100:00100000:3.0:1789760756.416986:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737683752115 time 737732289377 state 0x0 ref 1 00000100:00100000:3.0:1789760756.417001:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507280768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760756.417066:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20738048, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:3.0:1789760756.417077:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.417097:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760756.417103:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.417110:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.417126:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.447575:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972383 is committed 00002000:00010000:3.0:1789760756.447794:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20738048 -> 20742144 00010000:00010000:3.0:1789760756.447815:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447830:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447841:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447850:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760756.447855:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447866:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447891:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.447900:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b86ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.448029:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507280768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31023us (31600us total) trans 4294972383 rc 0/0 00000100:00100000:3.0:1789760756.448041:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5093 00000100:00100000:3.0:1789760756.448054:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737683752115 time 737732289377 state 0x2 ref 0 00000100:00100000:2.0:1789760756.452051:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.452130:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507280896 00000100:00100000:1.0:1789760756.452144:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507280896 00000100:00100000:1.0:1789760756.452212:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737683752115 time 737732289377 state 0x0 ref 1 00000100:00100000:1.0:1789760756.452229:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737732549793 time 737732289377 state 0x1 ref 1 00000100:00100000:1.0:1789760756.452308:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737732549793 time 737767610934 state 0x0 ref 1 00000100:00100000:1.0:1789760756.452322:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507280896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.452387:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20742144, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.452400:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.452420:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760756.452425:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.452433:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.452450:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.477031:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972384 is committed 00002000:00010000:2.0:1789760756.477924:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20742144 -> 20746240 00010000:00010000:2.0:1789760756.477949:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.477973:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.477984:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.477992:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760756.477999:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.478011:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.478040:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.478050:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b86b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.478202:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507280896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25872us (26154us total) trans 4294972384 rc 0/0 00000100:00100000:2.0:1789760756.478215:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5094 00000100:00100000:2.0:1789760756.478229:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737732549793 time 737767610934 state 0x2 ref 0 00000100:00100000:0.0:1789760756.483990:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.484052:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281024 00000100:00100000:1.0:1789760756.484065:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281024 00000100:00100000:1.0:1789760756.484139:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737732549793 time 737767610934 state 0x0 ref 1 00000100:00100000:1.0:1789760756.484160:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737767871350 time 737767610934 state 0x1 ref 1 00000100:00100000:1.0:1789760756.484196:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737767871350 time 737799498884 state 0x0 ref 1 00000100:00100000:1.0:1789760756.484211:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507281024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.484277:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20746240, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.484288:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.484307:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760756.484312:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.484319:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.484336:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.514469:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972385 is committed 00002000:00010000:1.0:1789760756.514612:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:1.0:1789760756.514633:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514648:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514658:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514665:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.514671:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514690:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514812:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.514820:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.514989:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507281024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30773us (31007us total) trans 4294972385 rc 0/0 00000100:00100000:1.0:1789760756.515003:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5095 00000100:00100000:1.0:1789760756.515014:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737767871350 time 737799498884 state 0x2 ref 0 00000100:00100000:3.0:1789760756.518429:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760756.518872:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281152 00000100:00100000:0.0:1789760756.518884:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281152 00000100:00100000:0.0:1789760756.518960:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737767871350 time 737799498884 state 0x0 ref 1 00000100:00100000:0.0:1789760756.518980:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737799759300 time 737799498884 state 0x1 ref 1 00000100:00100000:0.0:1789760756.519012:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737799759300 time 737834316199 state 0x0 ref 1 00000100:00100000:0.0:1789760756.519025:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760756.519080:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20750336, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:0.0:1789760756.519088:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.519104:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760756.519108:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.519114:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.519136:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.549484:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972386 is committed 00002000:00010000:1.0:1789760756.549621:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20750336 -> 20754432 00010000:00010000:1.0:1789760756.549644:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549660:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549670:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549688:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.549844:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549858:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549885:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.549899:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b86c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.550021:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30990us (31599us total) trans 4294972386 rc 0/0 00000100:00100000:1.0:1789760756.550034:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5096 00000100:00100000:1.0:1789760756.550047:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737799759300 time 737834316199 state 0x2 ref 0 00000100:00100000:3.0:1789760756.569139:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760756.569290:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281280 00000100:00100000:2.0:1789760756.569304:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281280 00000100:00100000:2.0:1789760756.569506:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737799759300 time 737834316199 state 0x0 ref 1 00000100:00100000:2.0:1789760756.569526:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737834576615 time 737834316199 state 0x1 ref 1 00000100:00100000:2.0:1789760756.569564:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737834576615 time 737884866870 state 0x0 ref 1 00000100:00100000:2.0:1789760756.569578:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507281280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760756.569640:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20754432, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760756.569651:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569670:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760756.569685:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.570814:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.570846:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.636360:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760756.638258:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972387 is committed 00002000:00010000:2.0:1789760756.638423:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20754432 -> 20758528 00010000:00010000:2.0:1789760756.638444:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638459:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638474:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638482:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760756.638488:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638498:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638525:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.638533:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.638671:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507281280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69088us (69540us total) trans 4294972387 rc 0/0 00000100:00100000:2.0:1789760756.638977:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5097 00000100:00100000:2.0:1789760756.638991:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737834576615 time 737884866870 state 0x2 ref 0 00000100:00100000:0.0:1789760756.647299:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.647430:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281408 00000100:00100000:1.0:1789760756.647445:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281408 00000100:00100000:1.0:1789760756.647528:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737834576615 time 737884866870 state 0x0 ref 1 00000100:00100000:1.0:1789760756.647548:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737885127286 time 737884866870 state 0x1 ref 1 00000100:00100000:1.0:1789760756.647590:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737885127286 time 737962891737 state 0x0 ref 1 00000100:00100000:1.0:1789760756.647606:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.647744:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20758528, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.647756:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.647778:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760756.647784:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.647792:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.647813:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.680594:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972388 is committed 00002000:00010000:0.0:1789760756.681366:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20758528 -> 20762624 00010000:00010000:0.0:1789760756.681389:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681406:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681417:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681426:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760756.681432:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681442:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681472:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.681481:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b86cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.681763:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34034us (34359us total) trans 4294972388 rc 0/0 00000100:00100000:0.0:1789760756.681776:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5098 00000100:00100000:0.0:1789760756.681787:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737885127286 time 737962891737 state 0x2 ref 0 00000100:00100000:0.0:1789760756.686424:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.686491:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281536 00000100:00100000:1.0:1789760756.686504:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281536 00000100:00100000:1.0:1789760756.686568:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737885127286 time 737962891737 state 0x0 ref 1 00000100:00100000:1.0:1789760756.686585:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737963152153 time 737962891737 state 0x1 ref 1 00000100:00100000:1.0:1789760756.686611:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737963152153 time 738001914823 state 0x0 ref 1 00000100:00100000:1.0:1789760756.686621:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507281536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.686687:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20762624, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.686782:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.686802:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760756.686807:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.686817:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.686839:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.715595:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:0.0:1789760756.715624:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715642:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715653:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715661:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760756.715666:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715786:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715816:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.715825:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b86d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.715945:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507281536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29317us (29529us total) trans 4294972389 rc 0/0 00000100:00100000:0.0:1789760756.715958:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5099 00000100:00100000:0.0:1789760756.715971:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737963152153 time 738001914823 state 0x2 ref 0 00000001:00080000:2.0:1789760756.716244:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972389 is committed 00000100:00100000:3.0:1789760756.724959:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760756.725107:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281664 00000100:00100000:2.0:1789760756.725121:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281664 00000100:00100000:2.0:1789760756.725197:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 737963152153 time 738001914823 state 0x0 ref 1 00000100:00100000:2.0:1789760756.725216:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738002175239 time 738001914823 state 0x1 ref 1 00000100:00100000:2.0:1789760756.725255:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738002175239 time 738040557665 state 0x0 ref 1 00000100:00100000:2.0:1789760756.725270:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760756.725333:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20766720, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760756.725346:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.725366:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760756.725371:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.725378:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.725395:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.773904:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972390 is committed 00002000:00010000:3.0:1789760756.776335:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20766720 -> 20770816 00010000:00010000:3.0:1789760756.776363:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776380:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776390:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776396:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760756.776401:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776419:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776442:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.776450:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b86dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.776598:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51322us (51645us total) trans 4294972390 rc 0/0 00000100:00100000:3.0:1789760756.776609:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5100 00000100:00100000:3.0:1789760756.776621:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738002175239 time 738040557665 state 0x2 ref 0 00000100:00100000:3.0:1789760756.781945:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760756.782510:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281792 00000100:00100000:0.0:1789760756.782523:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281792 00000100:00100000:0.0:1789760756.782608:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738002175239 time 738040557665 state 0x0 ref 1 00000100:00100000:0.0:1789760756.782632:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738040818081 time 738040557665 state 0x1 ref 1 00000100:00100000:0.0:1789760756.782936:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738040818081 time 738098237631 state 0x0 ref 1 00000100:00100000:0.0:1789760756.782952:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760756.783019:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20770816, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:0.0:1789760756.783031:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.783051:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760756.783057:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.783064:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.783080:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.829609:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972391 is committed 00002000:00010000:1.0:1789760756.829983:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:1.0:1789760756.830005:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830021:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830032:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830040:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.830046:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830056:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830083:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.830091:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b86e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.830454:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507281792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47497us (48519us total) trans 4294972391 rc 0/0 00000100:00100000:1.0:1789760756.830468:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5101 00000100:00100000:1.0:1789760756.830480:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738040818081 time 738098237631 state 0x2 ref 0 00000100:00100000:0.0:1789760756.834805:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.834904:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507281920 00000100:00100000:1.0:1789760756.834918:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507281920 00000100:00100000:1.0:1789760756.835000:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738040818081 time 738098237631 state 0x0 ref 1 00000100:00100000:1.0:1789760756.835019:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738098498047 time 738098237631 state 0x1 ref 1 00000100:00100000:1.0:1789760756.835095:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738098498047 time 738150396773 state 0x0 ref 1 00000100:00100000:1.0:1789760756.835111:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507281920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.835189:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20774912, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.835200:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.835220:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760756.835226:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.835242:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.835261:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.871501:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20774912 -> 20779008 00010000:00010000:1.0:1789760756.871528:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.871544:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.871556:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.871565:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760756.871570:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.871580:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.871580:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00010000:00010000:1.0:1789760756.871603:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.871612:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.871821:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507281920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36706us (37024us total) trans 4294972392 rc 0/0 00000100:00100000:1.0:1789760756.871833:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5102 00000100:00100000:1.0:1789760756.871844:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738098498047 time 738150396773 state 0x2 ref 0 00000100:00100000:3.0:1789760756.876211:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760756.876415:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282048 00000100:00100000:2.0:1789760756.876428:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282048 00000100:00100000:2.0:1789760756.876510:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738098498047 time 738150396773 state 0x0 ref 1 00000100:00100000:2.0:1789760756.876530:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738150657189 time 738150396773 state 0x1 ref 1 00000100:00100000:2.0:1789760756.876578:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738150657189 time 738191879522 state 0x0 ref 1 00000100:00100000:2.0:1789760756.876594:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507282048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760756.876661:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20779008, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760756.876684:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.876745:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760756.876751:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.876758:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.876776:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.930363:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972393 is committed 00002000:00010000:2.0:1789760756.930884:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20779008 -> 20783104 00010000:00010000:2.0:1789760756.930906:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930920:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930930:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930937:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760756.930942:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930952:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930976:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.930984:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b86f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.931241:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507282048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54643us (55119us total) trans 4294972393 rc 0/0 00000100:00100000:2.0:1789760756.931254:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5103 00000100:00100000:2.0:1789760756.931267:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738150657189 time 738191879522 state 0x2 ref 0 00000100:00100000:3.0:1789760756.938614:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.938746:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282176 00000100:00100000:1.0:1789760756.938760:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282176 00000100:00100000:1.0:1789760756.938844:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738150657189 time 738191879522 state 0x0 ref 1 00000100:00100000:1.0:1789760756.938865:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738192139938 time 738191879522 state 0x1 ref 1 00000100:00100000:1.0:1789760756.938903:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738192139938 time 738254205994 state 0x0 ref 1 00000100:00100000:1.0:1789760756.938919:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.938991:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20783104, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.939002:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.939023:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760756.939028:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.939035:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.939054:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.969133:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00002000:00010000:2.0:1789760756.969242:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20783104 -> 20787200 00010000:00010000:2.0:1789760756.969263:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969280:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969291:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969299:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760756.969304:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969320:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969345:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.969354:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b86f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.969513:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30590us (30906us total) trans 4294972394 rc 0/0 00000100:00100000:2.0:1789760756.969528:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5104 00000100:00100000:2.0:1789760756.969540:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738192139938 time 738254205994 state 0x2 ref 0 00000100:00100000:0.0:1789760756.973420:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760756.973559:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282304 00000100:00100000:1.0:1789760756.973574:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282304 00000100:00100000:1.0:1789760756.973654:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738192139938 time 738254205994 state 0x0 ref 1 00000100:00100000:1.0:1789760756.973684:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738254466410 time 738254205994 state 0x1 ref 1 00000100:00100000:1.0:1789760756.973798:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738254466410 time 738289099393 state 0x0 ref 1 00000100:00100000:1.0:1789760756.973812:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760756.973893:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20787200, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:1.0:1789760756.973904:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.973924:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760756.973930:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.973939:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.973956:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.020379:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00002000:00010000:1.0:1789760757.020595:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20787200 -> 20791296 00010000:00010000:1.0:1789760757.020617:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020916:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020928:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020937:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760757.020943:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020953:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020979:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.020987:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8700 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.021349:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47531us (47942us total) trans 4294972395 rc 0/0 00000100:00100000:0.0:1789760757.021364:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5105 00000100:00100000:0.0:1789760757.021375:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738254466410 time 738289099393 state 0x2 ref 0 00000100:00100000:3.0:1789760757.028690:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760757.029016:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282432 00000100:00100000:2.0:1789760757.029029:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282432 00000100:00100000:2.0:1789760757.029111:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738254466410 time 738289099393 state 0x0 ref 1 00000100:00100000:2.0:1789760757.029143:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738289359809 time 738289099393 state 0x1 ref 1 00000100:00100000:2.0:1789760757.029179:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738289359809 time 738344482046 state 0x0 ref 1 00000100:00100000:2.0:1789760757.029197:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507282432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760757.029260:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20791296, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xa007 00010000:00010000:2.0:1789760757.029270:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.029290:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760757.029295:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.029303:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.029318:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760757.033045:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760757.091267:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972396 is committed 00002000:00010000:3.0:1789760757.091980:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20791296 -> 20795392 00002000:00010000:3.0:1789760757.091987:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:3.0:1789760757.091991:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:3.0:1789760757.092008:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092023:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092032:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092040:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.092047:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092058:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092083:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.092096:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8707 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.092894:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507282432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63691us (64218us total) trans 4294972396 rc 0/0 00000100:00100000:3.0:1789760757.092908:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5106 00000100:00100000:3.0:1789760757.092920:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738289359809 time 738344482046 state 0x2 ref 0 00000100:00100000:2.0:1789760757.100826:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760757.100988:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282560 00000100:00100000:1.0:1789760757.101001:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282560 00000100:00100000:1.0:1789760757.101075:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738289359809 time 738344482046 state 0x0 ref 1 00000100:00100000:1.0:1789760757.101094:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738344742462 time 738344482046 state 0x1 ref 1 00000100:00100000:1.0:1789760757.101137:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738344742462 time 738416439569 state 0x0 ref 1 00000100:00100000:1.0:1789760757.101152:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507282560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760757.101219:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20795392, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760757.101229:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.101248:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760757.101253:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.101260:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.101283:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.139886:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:3.0:1789760757.139914:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139929:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139940:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139947:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.139952:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139962:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139992:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.140000:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b870e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.140167:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507282560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39010us (39347us total) trans 4294972397 rc 0/0 00000100:00100000:3.0:1789760757.140187:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5107 00000100:00100000:3.0:1789760757.140199:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738344742462 time 738416439569 state 0x2 ref 0 00000001:00080000:0.0:1789760757.140342:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972397 is committed 00000100:00100000:1.0:1789760757.146395:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760757.146489:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282688 00000100:00100000:0.0:1789760757.146504:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282688 00000100:00100000:0.0:1789760757.146576:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738344742462 time 738416439569 state 0x0 ref 1 00000100:00100000:0.0:1789760757.146596:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738416699985 time 738416439569 state 0x1 ref 1 00000100:00100000:0.0:1789760757.146627:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738416699985 time 738461930420 state 0x0 ref 1 00000100:00100000:0.0:1789760757.146642:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507282688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760757.146774:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20799488, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:0.0:1789760757.146786:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.146806:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760757.146811:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.146819:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.146835:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.185961:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972398 is committed 00002000:00010000:3.0:1789760757.186093:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20799488 -> 20803584 00010000:00010000:3.0:1789760757.186113:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186128:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186137:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186144:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.186150:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186164:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186200:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.186208:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8715 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.186334:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507282688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39685us (39945us total) trans 4294972398 rc 0/0 00000100:00100000:3.0:1789760757.186348:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5108 00000100:00100000:3.0:1789760757.186360:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738416699985 time 738461930420 state 0x2 ref 0 00000100:00100000:2.0:1789760757.190080:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.190239:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282816 00000100:00100000:3.0:1789760757.190254:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282816 00000100:00100000:3.0:1789760757.190329:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738416699985 time 738461930420 state 0x0 ref 1 00000100:00100000:3.0:1789760757.190349:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738462190836 time 738461930420 state 0x1 ref 1 00000100:00100000:3.0:1789760757.190391:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738462190836 time 738505694171 state 0x0 ref 1 00000100:00100000:3.0:1789760757.190404:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.190476:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20803584, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.190487:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.190507:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.190512:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.190520:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.190539:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.227569:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00002000:00010000:3.0:1789760757.227758:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20803584 -> 20807680 00010000:00010000:3.0:1789760757.227779:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227794:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227805:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227812:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.227818:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227830:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227853:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.227862:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b871c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.227971:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507282816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37562us (37899us total) trans 4294972399 rc 0/0 00000100:00100000:3.0:1789760757.227981:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5109 00000100:00100000:3.0:1789760757.227993:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738462190836 time 738505694171 state 0x2 ref 0 00000100:00100000:2.0:1789760757.231329:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.231428:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507282944 00000100:00100000:3.0:1789760757.231440:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507282944 00000100:00100000:3.0:1789760757.231516:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738462190836 time 738505694171 state 0x0 ref 1 00000100:00100000:3.0:1789760757.231535:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738505954587 time 738505694171 state 0x1 ref 1 00000100:00100000:3.0:1789760757.231629:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738505954587 time 738546930500 state 0x0 ref 1 00000100:00100000:3.0:1789760757.231644:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507282944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.231764:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20807680, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.231775:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.231796:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.231802:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.231810:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.231834:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.275182:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20807680 -> 20811776 00010000:00010000:3.0:1789760757.275209:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275225:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275236:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275244:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.275251:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275262:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275287:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275296:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8723 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.275425:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507282944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43776us (44104us total) trans 4294972400 rc 0/0 00000100:00100000:3.0:1789760757.275437:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5110 00000100:00100000:3.0:1789760757.275449:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738505954587 time 738546930500 state 0x2 ref 0 00000001:00080000:0.0:1789760757.276113:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00000100:00100000:0.0:1789760757.281105:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.281194:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283072 00000100:00100000:3.0:1789760757.281207:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283072 00000100:00100000:3.0:1789760757.281282:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738505954587 time 738546930500 state 0x0 ref 1 00000100:00100000:3.0:1789760757.281302:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738547190916 time 738546930500 state 0x1 ref 1 00000100:00100000:3.0:1789760757.281337:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738547190916 time 738596640498 state 0x0 ref 1 00000100:00100000:3.0:1789760757.281351:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.281417:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20811776, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.281428:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.281450:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.281455:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.281463:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.281481:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.344897:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972401 is committed 00002000:00010000:3.0:1789760757.345111:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:3.0:1789760757.345131:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345146:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345156:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345164:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.345170:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345179:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345204:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.345212:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b872a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.348348:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66980us (67238us total) trans 4294972401 rc 0/0 00000100:00100000:3.0:1789760757.348373:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5111 00000100:00100000:3.0:1789760757.348387:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738547190916 time 738596640498 state 0x2 ref 0 00000100:00100000:2.0:1789760757.352645:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.352834:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283200 00000100:00100000:3.0:1789760757.352849:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283200 00000100:00100000:3.0:1789760757.352932:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738547190916 time 738596640498 state 0x0 ref 1 00000100:00100000:3.0:1789760757.352952:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738596900914 time 738596640498 state 0x1 ref 1 00000100:00100000:3.0:1789760757.352993:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738596900914 time 738668295901 state 0x0 ref 1 00000100:00100000:3.0:1789760757.353013:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.353090:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20815872, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.353100:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.353122:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.353128:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.353136:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.353153:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.394734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972402 is committed 00002000:00010000:3.0:1789760757.394917:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:3.0:1789760757.394939:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.394956:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.394966:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.395164:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.395240:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.395252:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.395276:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.395283:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8731 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.395458:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42441us (42824us total) trans 4294972402 rc 0/0 00000100:00100000:3.0:1789760757.395472:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5112 00000100:00100000:3.0:1789760757.395484:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738596900914 time 738668295901 state 0x2 ref 0 00000100:00100000:2.0:1789760757.399549:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.399976:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283328 00000100:00100000:3.0:1789760757.399992:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283328 00000100:00100000:3.0:1789760757.400069:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738596900914 time 738668295901 state 0x0 ref 1 00000100:00100000:3.0:1789760757.400088:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738668556317 time 738668295901 state 0x1 ref 1 00000100:00100000:3.0:1789760757.400257:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738668556317 time 738715417974 state 0x0 ref 1 00000100:00100000:3.0:1789760757.400275:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507283328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.400341:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20819968, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.400353:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.400373:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.400377:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.400386:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.400618:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.445572:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972403 is committed 00002000:00010000:1.0:1789760757.445816:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20819968 -> 20824064 00010000:00010000:1.0:1789760757.445837:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445852:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445863:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445871:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760757.445878:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445889:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445913:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.445921:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8738 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.446086:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507283328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45807us (46544us total) trans 4294972403 rc 0/0 00000100:00100000:1.0:1789760757.446099:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5113 00000100:00100000:1.0:1789760757.446111:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738668556317 time 738715417974 state 0x2 ref 0 00000100:00100000:0.0:1789760757.455104:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.455227:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283456 00000100:00100000:3.0:1789760757.455241:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283456 00000100:00100000:3.0:1789760757.455316:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738668556317 time 738715417974 state 0x0 ref 1 00000100:00100000:3.0:1789760757.455335:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738715678390 time 738715417974 state 0x1 ref 1 00000100:00100000:3.0:1789760757.455458:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738715678390 time 738770759431 state 0x0 ref 1 00000100:00100000:3.0:1789760757.455477:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507283456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.455554:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20824064, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.455565:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.455585:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.455590:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.455597:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.455614:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.484041:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972404 is committed 00002000:00010000:0.0:1789760757.484053:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20824064 -> 20828160 00010000:00010000:0.0:1789760757.484078:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484094:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484111:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484118:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760757.484124:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484134:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484158:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.484166:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b873f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.484294:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507283456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28813us (29199us total) trans 4294972404 rc 0/0 00000100:00100000:0.0:1789760757.484305:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5114 00000100:00100000:0.0:1789760757.484316:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738715678390 time 738770759431 state 0x2 ref 0 00000100:00100000:2.0:1789760757.490206:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.490440:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283584 00000100:00100000:3.0:1789760757.490454:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283584 00000100:00100000:3.0:1789760757.490507:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738715678390 time 738770759431 state 0x0 ref 1 00000100:00100000:3.0:1789760757.490522:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738771019847 time 738770759431 state 0x1 ref 1 00000100:00100000:3.0:1789760757.490552:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738771019847 time 738805855311 state 0x0 ref 1 00000100:00100000:3.0:1789760757.490565:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507283584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.490609:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20828160, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.490618:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.490640:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.490644:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.490650:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.490664:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.542473:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972405 is committed 00002000:00010000:2.0:1789760757.542571:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20828160 -> 20832256 00010000:00010000:2.0:1789760757.542586:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542598:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542607:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542617:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760757.542622:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542631:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542651:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.542658:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8746 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.542932:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507283584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52365us (52737us total) trans 4294972405 rc 0/0 00000100:00100000:2.0:1789760757.542943:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5115 00000100:00100000:2.0:1789760757.542953:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738771019847 time 738805855311 state 0x2 ref 0 00000100:00100000:2.0:1789760757.546188:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760757.546307:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283712 00000100:00100000:1.0:1789760757.546318:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283712 00000100:00100000:1.0:1789760757.546365:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738771019847 time 738805855311 state 0x0 ref 1 00000100:00100000:1.0:1789760757.546378:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738806115727 time 738805855311 state 0x1 ref 1 00000100:00100000:1.0:1789760757.546402:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738806115727 time 738861705995 state 0x0 ref 1 00000100:00100000:1.0:1789760757.546414:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507283712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760757.546463:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20832256, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760757.546473:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.546490:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760757.546494:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.546499:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.546510:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.576940:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972406 is committed 00002000:00010000:1.0:1789760757.577095:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20832256 -> 20836352 00010000:00010000:1.0:1789760757.577110:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577121:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577130:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577137:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760757.577142:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577151:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577171:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.577178:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b874d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.577264:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507283712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30847us (31082us total) trans 4294972406 rc 0/0 00000100:00100000:1.0:1789760757.577277:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5116 00000100:00100000:1.0:1789760757.577287:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738806115727 time 738861705995 state 0x2 ref 0 00000100:00100000:0.0:1789760757.589226:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.589392:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283840 00000100:00100000:3.0:1789760757.589402:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283840 00000100:00100000:3.0:1789760757.589460:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738806115727 time 738861705995 state 0x0 ref 1 00000100:00100000:3.0:1789760757.589476:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738861966411 time 738861705995 state 0x1 ref 1 00000100:00100000:3.0:1789760757.589505:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738861966411 time 738904808337 state 0x0 ref 1 00000100:00100000:3.0:1789760757.589517:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.589564:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20836352, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.589572:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.589586:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.589592:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.589599:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.589619:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.620780:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972407 is committed 00002000:00010000:0.0:1789760757.620960:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20836352 -> 20840448 00010000:00010000:0.0:1789760757.620976:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.620990:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.620999:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.621006:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760757.621010:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.621019:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.621037:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.621044:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8754 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.621225:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31705us (32003us total) trans 4294972407 rc 0/0 00000100:00100000:0.0:1789760757.621235:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5117 00000100:00100000:0.0:1789760757.621243:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738861966411 time 738904808337 state 0x2 ref 0 00000100:00100000:2.0:1789760757.624552:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.624636:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507283968 00000100:00100000:3.0:1789760757.624647:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507283968 00000100:00100000:3.0:1789760757.624760:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738861966411 time 738904808337 state 0x0 ref 1 00000100:00100000:3.0:1789760757.624775:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738905068753 time 738904808337 state 0x1 ref 1 00000100:00100000:3.0:1789760757.624794:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738905068753 time 738940098210 state 0x0 ref 1 00000100:00100000:3.0:1789760757.624806:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507283968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.624855:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20840448, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.624864:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.624879:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760757.624884:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.624890:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.624905:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.659821:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760757.660782:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20840448 -> 20844544 00010000:00010000:1.0:1789760757.660798:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660811:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660819:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660827:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760757.660832:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660846:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660864:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.660872:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b875b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.661025:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507283968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36155us (36418us total) trans 4294972408 rc 0/0 00000100:00100000:1.0:1789760757.661035:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5118 00000100:00100000:1.0:1789760757.661044:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738905068753 time 738940098210 state 0x2 ref 0 00000001:00080000:1.0:1789760757.661081:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972408 is committed 00000100:00100000:2.0:1789760757.669748:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760757.670776:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284096 00000100:00100000:1.0:1789760757.670787:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284096 00000100:00100000:1.0:1789760757.670843:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738905068753 time 738940098210 state 0x0 ref 1 00000100:00100000:1.0:1789760757.670859:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738940358626 time 738940098210 state 0x1 ref 1 00000100:00100000:1.0:1789760757.670889:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738940358626 time 738986192267 state 0x0 ref 1 00000100:00100000:1.0:1789760757.670901:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760757.670948:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20844544, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760757.670957:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.670983:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760757.670988:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.670994:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.671009:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.707170:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20844544 -> 20848640 00010000:00010000:1.0:1789760757.707191:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707203:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707212:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707219:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760757.707224:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707233:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707254:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.707262:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8762 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.707359:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507284096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36455us (37615us total) trans 4294972409 rc 0/0 00000100:00100000:1.0:1789760757.707374:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5119 00000100:00100000:1.0:1789760757.707384:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738940358626 time 738986192267 state 0x2 ref 0 00000001:00080000:3.0:1789760757.708354:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972409 is committed 00000100:00100000:0.0:1789760757.710821:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760757.710913:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284224 00000100:00100000:3.0:1789760757.710922:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284224 00000100:00100000:3.0:1789760757.710973:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738940358626 time 738986192267 state 0x0 ref 1 00000100:00100000:3.0:1789760757.710988:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738986452683 time 738986192267 state 0x1 ref 1 00000100:00100000:3.0:1789760757.711050:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738986452683 time 739026353033 state 0x0 ref 1 00000100:00100000:3.0:1789760757.711063:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760757.711109:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20848640, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:3.0:1789760757.711117:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.711132:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760757.711136:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.711142:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.711156:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.767920:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972410 is committed 00002000:00010000:2.0:1789760757.768330:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20848640 -> 20852736 00010000:00010000:2.0:1789760757.768352:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768382:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768440:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768448:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760757.768454:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768465:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768492:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.768501:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8769 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.768680:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57600us (57847us total) trans 4294972410 rc 0/0 00000100:00100000:2.0:1789760757.768741:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5120 00000100:00100000:2.0:1789760757.768754:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738986452683 time 739026353033 state 0x2 ref 0 00000100:00100000:2.0:1789760757.781860:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760757.782859:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284352 00000100:00100000:1.0:1789760757.782877:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284352 00000100:00100000:1.0:1789760757.782957:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 738986452683 time 739026353033 state 0x0 ref 1 00000100:00100000:1.0:1789760757.782978:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739026613449 time 739026353033 state 0x1 ref 1 00000100:00100000:1.0:1789760757.783024:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739026613449 time 739098326397 state 0x0 ref 1 00000100:00100000:1.0:1789760757.783039:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760757.783107:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20852736, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760757.783117:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.783136:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760757.783142:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.783149:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.783166:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.885423:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972411 is committed 00002000:00010000:2.0:1789760757.885526:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20852736 -> 20856832 00010000:00010000:2.0:1789760757.885552:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885572:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885583:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885592:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760757.885597:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885607:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885632:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.885639:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8770 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.885923:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 102869us (104059us total) trans 4294972411 rc 0/0 00000100:00100000:1.0:1789760757.885940:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5121 00000100:00100000:1.0:1789760757.885952:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739026613449 time 739098326397 state 0x2 ref 0 00000100:00100000:3.0:1789760757.890656:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760757.890911:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284480 00000100:00100000:1.0:1789760757.890922:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284480 00000100:00100000:1.0:1789760757.890975:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739026613449 time 739098326397 state 0x0 ref 1 00000100:00100000:1.0:1789760757.890991:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739098586813 time 739098326397 state 0x1 ref 1 00000100:00100000:1.0:1789760757.891022:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739098586813 time 739206325720 state 0x0 ref 1 00000100:00100000:1.0:1789760757.891037:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760757.891085:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20856832, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760757.891094:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.891109:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760757.891124:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.891132:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.891147:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.952981:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972412 is committed 00002000:00010000:3.0:1789760757.953052:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20856832 -> 20860928 00010000:00010000:3.0:1789760757.953066:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953079:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953088:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953095:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760757.953100:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953109:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953130:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.953137:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8777 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.953242:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62204us (62591us total) trans 4294972412 rc 0/0 00000100:00100000:3.0:1789760757.953254:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5122 00000100:00100000:3.0:1789760757.953264:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739098586813 time 739206325720 state 0x2 ref 0 00000100:00100000:0.0:1789760757.957260:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760757.958045:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284608 00000100:00100000:2.0:1789760757.958056:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284608 00000100:00100000:2.0:1789760757.958112:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739098586813 time 739206325720 state 0x0 ref 1 00000100:00100000:2.0:1789760757.958127:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739206586136 time 739206325720 state 0x1 ref 1 00000100:00100000:2.0:1789760757.958161:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739206586136 time 739273464148 state 0x0 ref 1 00000100:00100000:2.0:1789760757.958173:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760757.958219:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20860928, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:2.0:1789760757.958228:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.958242:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760757.958245:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.958251:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.958265:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.000458:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972413 is committed 00002000:00010000:3.0:1789760758.000611:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:3.0:1789760758.000626:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000638:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000645:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000652:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.000657:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000676:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000835:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.000845:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b877e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.000930:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42754us (43674us total) trans 4294972413 rc 0/0 00000100:00100000:3.0:1789760758.000940:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5123 00000100:00100000:3.0:1789760758.000950:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739206586136 time 739273464148 state 0x2 ref 0 00000100:00100000:0.0:1789760758.005836:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.005922:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284736 00000100:00100000:1.0:1789760758.005933:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284736 00000100:00100000:1.0:1789760758.005988:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739206586136 time 739273464148 state 0x0 ref 1 00000100:00100000:1.0:1789760758.006006:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739273724564 time 739273464148 state 0x1 ref 1 00000100:00100000:1.0:1789760758.006036:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739273724564 time 739321339385 state 0x0 ref 1 00000100:00100000:1.0:1789760758.006049:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.006095:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20865024, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xa007 00010000:00010000:1.0:1789760758.006105:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.006132:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.006137:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.006144:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.006158:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.046913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972414 is committed 00002000:00010000:3.0:1789760758.047029:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20865024 -> 20869120 00002000:00010000:3.0:1789760758.047033:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:3.0:1789760758.047036:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:3.0:1789760758.047049:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047062:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047072:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047080:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.047085:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047095:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047115:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.047122:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8785 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.047213:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41160us (41383us total) trans 4294972414 rc 0/0 00000100:00100000:3.0:1789760758.047223:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5124 00000100:00100000:3.0:1789760758.047232:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739273724564 time 739321339385 state 0x2 ref 0 00000100:00100000:1.0:1789760758.058394:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.058487:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284864 00000100:00100000:3.0:1789760758.058497:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284864 00000100:00100000:3.0:1789760758.058550:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739273724564 time 739321339385 state 0x0 ref 1 00000100:00100000:3.0:1789760758.058567:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739321599801 time 739321339385 state 0x1 ref 1 00000100:00100000:3.0:1789760758.058602:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739321599801 time 739373905475 state 0x0 ref 1 00000100:00100000:3.0:1789760758.058616:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760758.058673:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20869120, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:3.0:1789760758.058683:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.058850:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760758.058854:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.058861:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.058877:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.109159:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20869120 -> 20873216 00010000:00010000:2.0:1789760758.109179:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109191:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109201:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109215:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.109221:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109232:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109252:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.109261:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b878c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.109469:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507284864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50849us (51079us total) trans 4294972415 rc 0/0 00000100:00100000:2.0:1789760758.109481:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5125 00000100:00100000:2.0:1789760758.109491:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739321599801 time 739373905475 state 0x2 ref 0 00000001:00080000:0.0:1789760758.109629:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972415 is committed 00000100:00100000:2.0:1789760758.119060:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.119141:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507284992 00000100:00100000:3.0:1789760758.119150:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507284992 00000100:00100000:3.0:1789760758.119207:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739321599801 time 739373905475 state 0x0 ref 1 00000100:00100000:3.0:1789760758.119224:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739374165891 time 739373905475 state 0x1 ref 1 00000100:00100000:3.0:1789760758.119256:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739374165891 time 739434559537 state 0x0 ref 1 00000100:00100000:3.0:1789760758.119270:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760758.119319:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20873216, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:3.0:1789760758.119328:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.119343:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760758.119348:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.119355:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.119370:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.166008:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:3.0:1789760758.166027:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166039:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166048:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166057:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.166062:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166072:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166091:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166100:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8793 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.166233:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972416 is committed 00000100:00100000:3.0:1789760758.166471:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507284992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47198us (47416us total) trans 4294972416 rc 0/0 00000100:00100000:3.0:1789760758.166482:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5126 00000100:00100000:3.0:1789760758.166491:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739374165891 time 739434559537 state 0x2 ref 0 00000100:00100000:3.0:1789760758.177450:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.177625:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285120 00000100:00100000:1.0:1789760758.177636:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285120 00000100:00100000:1.0:1789760758.177793:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739374165891 time 739434559537 state 0x0 ref 1 00000100:00100000:1.0:1789760758.177809:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739434819953 time 739434559537 state 0x1 ref 1 00000100:00100000:1.0:1789760758.177886:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739434819953 time 739493187892 state 0x0 ref 1 00000100:00100000:1.0:1789760758.177900:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507285120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.177947:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20877312, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.177956:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.177972:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.177977:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.177983:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.177998:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.212734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972417 is committed 00002000:00010000:1.0:1789760758.212844:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20877312 -> 20881408 00010000:00010000:1.0:1789760758.212860:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212871:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212880:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212887:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.212891:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212901:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212919:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.212927:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b879a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.213025:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507285120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35123us (35580us total) trans 4294972417 rc 0/0 00000100:00100000:1.0:1789760758.213034:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5127 00000100:00100000:1.0:1789760758.213044:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739434819953 time 739493187892 state 0x2 ref 0 00000100:00100000:0.0:1789760758.217681:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.217962:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285248 00000100:00100000:2.0:1789760758.217973:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285248 00000100:00100000:2.0:1789760758.218025:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739434819953 time 739493187892 state 0x0 ref 1 00000100:00100000:2.0:1789760758.218041:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739493448308 time 739493187892 state 0x1 ref 1 00000100:00100000:2.0:1789760758.218070:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739493448308 time 739533373494 state 0x0 ref 1 00000100:00100000:2.0:1789760758.218082:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507285248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760758.218131:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20881408, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:2.0:1789760758.218141:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.218157:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760758.218162:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.218168:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.218183:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.250790:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972418 is committed 00002000:00010000:1.0:1789760758.250945:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20881408 -> 20885504 00010000:00010000:1.0:1789760758.250961:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.250974:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.250984:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.250992:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.250998:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.251007:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.251127:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.251136:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b87a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.251228:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507285248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33142us (33563us total) trans 4294972418 rc 0/0 00000100:00100000:1.0:1789760758.251238:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5128 00000100:00100000:1.0:1789760758.251246:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739493448308 time 739533373494 state 0x2 ref 0 00000100:00100000:0.0:1789760758.259635:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.259731:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285376 00000100:00100000:3.0:1789760758.259742:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285376 00000100:00100000:3.0:1789760758.259796:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739493448308 time 739533373494 state 0x0 ref 1 00000100:00100000:3.0:1789760758.259811:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739533633910 time 739533373494 state 0x1 ref 1 00000100:00100000:3.0:1789760758.259888:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739533633910 time 739575190698 state 0x0 ref 1 00000100:00100000:3.0:1789760758.259903:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507285376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760758.259948:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20885504, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:3.0:1789760758.259956:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.259972:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760758.259976:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.259982:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.259996:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.286987:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20885504 -> 20889600 00010000:00010000:3.0:1789760758.287007:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287019:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287029:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287036:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.287041:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287050:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287070:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287078:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b87a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.287168:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507285376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27263us (27538us total) trans 4294972419 rc 0/0 00000100:00100000:3.0:1789760758.287178:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5129 00000100:00100000:3.0:1789760758.287188:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739533633910 time 739575190698 state 0x2 ref 0 00000001:00080000:3.0:1789760758.288236:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972419 is committed 00000100:00100000:3.0:1789760758.294309:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.294528:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285504 00000100:00100000:1.0:1789760758.294537:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285504 00000100:00100000:1.0:1789760758.294595:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739533633910 time 739575190698 state 0x0 ref 1 00000100:00100000:1.0:1789760758.294609:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739575451114 time 739575190698 state 0x1 ref 1 00000100:00100000:1.0:1789760758.294641:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739575451114 time 739609944085 state 0x0 ref 1 00000100:00100000:1.0:1789760758.294653:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.294861:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20889600, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.294872:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.294889:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.294894:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.294901:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.294916:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.325725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972420 is committed 00002000:00010000:3.0:1789760758.325893:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20889600 -> 20893696 00010000:00010000:3.0:1789760758.325907:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325920:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325928:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325940:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.325945:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325954:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325971:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.325979:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.326070:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31414us (31767us total) trans 4294972420 rc 0/0 00000100:00100000:3.0:1789760758.326081:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5130 00000100:00100000:3.0:1789760758.326090:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739575451114 time 739609944085 state 0x2 ref 0 00000100:00100000:0.0:1789760758.329847:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.329942:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285632 00000100:00100000:1.0:1789760758.329951:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285632 00000100:00100000:1.0:1789760758.330003:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739575451114 time 739609944085 state 0x0 ref 1 00000100:00100000:1.0:1789760758.330018:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739610204501 time 739609944085 state 0x1 ref 1 00000100:00100000:1.0:1789760758.330099:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739610204501 time 739645402000 state 0x0 ref 1 00000100:00100000:1.0:1789760758.330113:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.330162:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20893696, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.330171:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.330187:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.330193:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.330199:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.330215:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.392757:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00002000:00010000:1.0:1789760758.392905:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20893696 -> 20897792 00010000:00010000:1.0:1789760758.392920:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392932:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392941:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392949:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.392953:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392962:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392981:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.392988:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.393090:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62974us (63247us total) trans 4294972421 rc 0/0 00000100:00100000:1.0:1789760758.393099:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5131 00000100:00100000:1.0:1789760758.393108:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739610204501 time 739645402000 state 0x2 ref 0 00000100:00100000:0.0:1789760758.400008:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.400131:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285760 00000100:00100000:3.0:1789760758.400141:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285760 00000100:00100000:3.0:1789760758.400195:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739610204501 time 739645402000 state 0x0 ref 1 00000100:00100000:3.0:1789760758.400209:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739645662416 time 739645402000 state 0x1 ref 1 00000100:00100000:3.0:1789760758.400239:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739645662416 time 739715542057 state 0x0 ref 1 00000100:00100000:3.0:1789760758.400255:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507285760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760758.400307:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20897792, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:3.0:1789760758.400317:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.400332:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760758.400336:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.400343:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.400356:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.447124:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972422 is committed 00002000:00010000:2.0:1789760758.447306:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20897792 -> 20901888 00010000:00010000:2.0:1789760758.447327:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447343:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447355:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447363:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.447369:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447381:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447406:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.447414:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b87bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.447555:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507285760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47295us (47550us total) trans 4294972422 rc 0/0 00000100:00100000:2.0:1789760758.447568:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5132 00000100:00100000:2.0:1789760758.447580:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739645662416 time 739715542057 state 0x2 ref 0 00000100:00100000:3.0:1789760758.451435:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.451559:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507285888 00000100:00100000:1.0:1789760758.451572:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507285888 00000100:00100000:1.0:1789760758.451643:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739645662416 time 739715542057 state 0x0 ref 1 00000100:00100000:1.0:1789760758.451672:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739715802473 time 739715542057 state 0x1 ref 1 00000100:00100000:1.0:1789760758.451800:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739715802473 time 739767101817 state 0x0 ref 1 00000100:00100000:1.0:1789760758.451817:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.451893:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20901888, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.451904:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.451926:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.451934:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.451945:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.451963:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.485603:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760758.485791:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000001:00080000:3.0:1789760758.486952:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972423 is committed 00000100:00100000:0.0:1789760758.487366:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760758.487479:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.487503:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286016 00000100:00100000:2.0:1789760758.487515:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286016 00000100:00100000:2.0:1789760758.487573:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 474 00000100:00100000:2.0:1789760758.487586:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507286016:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760758.487689:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2418:x1876699507286016:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 106us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760758.487722:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 474 00000100:00100000:1.0:1789760758.487853:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286144 00000100:00100000:1.0:1789760758.487858:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286144 00000100:00100000:1.0:1789760758.487904:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286400 00000100:00100000:1.0:1789760758.487908:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286400 00000100:00100000:1.0:1789760758.487933:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 918 00000100:00100000:1.0:1789760758.487939:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507286144:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760758.487976:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507286144:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 37us (2387us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760758.487983:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 918 00000100:00100000:1.0:1789760758.487998:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 919 00000100:00100000:1.0:1789760758.488005:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507286400:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760758.488030:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507286400:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 26us (2240us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760758.488037:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 919 00002000:00010000:1.0:1789760758.488105:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20901888 -> 20905984 00010000:00010000:1.0:1789760758.488122:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488137:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488147:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488161:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.488166:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488254:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488279:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.488289:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.488336:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507285888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36523us (36916us total) trans 4294972423 rc 0/0 00000100:00100000:1.0:1789760758.488346:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5133 00000100:00100000:1.0:1789760758.488356:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739715802473 time 739767101817 state 0x2 ref 0 00000100:00080000:2.0:1789760758.488426:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760758.488687:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.489367:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286272 00000100:00100000:3.0:1789760758.489375:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286272 00000100:00080000:0.0:1789760758.489453:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760758.489485:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286528 00000100:00100000:2.0:1789760758.489490:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286528 00000100:00100000:2.0:1789760758.489571:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 920 00000100:00100000:2.0:1789760758.489581:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507286272:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760758.489640:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507286272:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 60us (2162us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760758.489647:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 920 00000100:00100000:2.0:1789760758.489675:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 921 00000100:00100000:2.0:1789760758.489683:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507286528:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760758.489741:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507286528:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 59us (1055us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760758.489748:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 921 00000100:00080000:0.0:1789760758.490406:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760758.496311:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.496478:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286656 00000100:00100000:1.0:1789760758.496491:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286656 00000100:00100000:1.0:1789760758.496558:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739715802473 time 739767101817 state 0x0 ref 1 00000100:00100000:1.0:1789760758.496573:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739767362233 time 739767101817 state 0x1 ref 1 00000100:00100000:1.0:1789760758.496626:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739767362233 time 739811930223 state 0x0 ref 1 00000100:00100000:1.0:1789760758.496640:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507286656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.496734:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20905984, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.496745:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.496765:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.496771:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.496779:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.496797:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.531626:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972424 is committed 00002000:00010000:0.0:1789760758.531765:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20905984 -> 20910080 00010000:00010000:0.0:1789760758.531785:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531800:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531810:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531904:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760758.531912:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531923:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531948:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.531957:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.532294:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507286656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35530us (35878us total) trans 4294972424 rc 0/0 00000100:00100000:1.0:1789760758.532307:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5134 00000100:00100000:1.0:1789760758.532319:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739767362233 time 739811930223 state 0x2 ref 0 00000100:00100000:3.0:1789760758.537169:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.537340:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286784 00000100:00100000:2.0:1789760758.537353:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286784 00000100:00100000:2.0:1789760758.537429:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739767362233 time 739811930223 state 0x0 ref 1 00000100:00100000:2.0:1789760758.537450:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739812190639 time 739811930223 state 0x1 ref 1 00000100:00100000:2.0:1789760758.537487:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739812190639 time 739852789714 state 0x0 ref 1 00000100:00100000:2.0:1789760758.537503:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507286784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760758.537569:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20910080, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:2.0:1789760758.537580:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.537600:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760758.537605:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.537613:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.537629:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.583957:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20910080 -> 20914176 00010000:00010000:3.0:1789760758.583987:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584004:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584014:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584022:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760758.584028:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584056:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584082:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.584091:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b87d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.584096:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972425 is committed 00000100:00100000:3.0:1789760758.584240:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507286784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46732us (47080us total) trans 4294972425 rc 0/0 00000100:00100000:3.0:1789760758.584252:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5135 00000100:00100000:3.0:1789760758.584264:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739812190639 time 739852789714 state 0x2 ref 0 00000100:00100000:0.0:1789760758.587600:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.587674:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507286912 00000100:00100000:1.0:1789760758.587688:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507286912 00000100:00100000:1.0:1789760758.587783:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739812190639 time 739852789714 state 0x0 ref 1 00000100:00100000:1.0:1789760758.587803:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739853050130 time 739852789714 state 0x1 ref 1 00000100:00100000:1.0:1789760758.587842:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739853050130 time 739903144770 state 0x0 ref 1 00000100:00100000:1.0:1789760758.587856:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507286912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.587996:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20914176, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.588007:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.588025:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.588031:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.588038:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.588056:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.630772:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:1.0:1789760758.630800:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630818:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630828:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630837:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.630844:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630855:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630880:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.630888:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b87d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.631022:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507286912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43148us (43416us total) trans 4294972426 rc 0/0 00000100:00100000:1.0:1789760758.631034:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5136 00000100:00100000:1.0:1789760758.631047:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739853050130 time 739903144770 state 0x2 ref 0 00000001:00080000:1.0:1789760758.631098:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972426 is committed 00000100:00100000:2.0:1789760758.634595:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.634774:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287040 00000100:00100000:1.0:1789760758.634787:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287040 00000100:00100000:1.0:1789760758.634864:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739853050130 time 739903144770 state 0x0 ref 1 00000100:00100000:1.0:1789760758.634885:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739903405186 time 739903144770 state 0x1 ref 1 00000100:00100000:1.0:1789760758.634924:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739903405186 time 739950226390 state 0x0 ref 1 00000100:00100000:1.0:1789760758.634939:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.635009:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20918272, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.635020:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.635040:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760758.635046:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.635054:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.635071:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.684594:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760758.690130:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972427 is committed 00002000:00010000:2.0:1789760758.690939:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:2.0:1789760758.690964:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.690981:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.690991:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.691000:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.691006:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.691016:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.691046:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.691054:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b87e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.691186:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56242us (56598us total) trans 4294972427 rc 0/0 00000100:00100000:2.0:1789760758.691200:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5137 00000100:00100000:2.0:1789760758.691212:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739903405186 time 739950226390 state 0x2 ref 0 00000100:00100000:2.0:1789760758.696282:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760758.696494:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287168 00000100:00100000:3.0:1789760758.696509:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287168 00000100:00100000:3.0:1789760758.696582:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739903405186 time 739950226390 state 0x0 ref 1 00000100:00100000:3.0:1789760758.696601:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739950486806 time 739950226390 state 0x1 ref 1 00000100:00100000:3.0:1789760758.696627:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739950486806 time 740011929945 state 0x0 ref 1 00000100:00100000:3.0:1789760758.696642:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760758.696911:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20922368, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:2.0:1789760758.696924:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.696945:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760758.696951:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.696959:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.696977:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.727064:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972428 is committed 00002000:00010000:2.0:1789760758.727249:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20922368 -> 20926464 00010000:00010000:2.0:1789760758.727270:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727286:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727304:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727312:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.727318:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727329:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727353:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.727361:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b87e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.727495:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30848us (31220us total) trans 4294972428 rc 0/0 00000100:00100000:2.0:1789760758.727507:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5138 00000100:00100000:2.0:1789760758.727519:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739950486806 time 740011929945 state 0x2 ref 0 00000100:00100000:0.0:1789760758.734038:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760758.734159:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287296 00000100:00100000:1.0:1789760758.734181:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287296 00000100:00100000:1.0:1789760758.734258:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 739950486806 time 740011929945 state 0x0 ref 1 00000100:00100000:1.0:1789760758.734279:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740012190361 time 740011929945 state 0x1 ref 1 00000100:00100000:1.0:1789760758.734309:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740012190361 time 740049611966 state 0x0 ref 1 00000100:00100000:1.0:1789760758.734323:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760758.734385:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20926464, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760758.734397:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.734423:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760758.734427:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.734440:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.734459:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.796302:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:1.0:1789760758.796332:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796351:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796362:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796370:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.796376:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796387:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796413:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.796422:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b87ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.796546:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507287296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62218us (62516us total) trans 4294972429 rc 0/0 00000100:00100000:1.0:1789760758.796557:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5139 00000100:00100000:1.0:1789760758.796580:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740012190361 time 740049611966 state 0x2 ref 0 00000001:00080000:3.0:1789760758.796681:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972429 is committed 00000100:00100000:1.0:1789760758.799757:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.799871:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287424 00000100:00100000:2.0:1789760758.799888:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287424 00000100:00100000:2.0:1789760758.799973:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740012190361 time 740049611966 state 0x0 ref 1 00000100:00100000:2.0:1789760758.799994:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740049872382 time 740049611966 state 0x1 ref 1 00000100:00100000:2.0:1789760758.800022:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740049872382 time 740115324673 state 0x0 ref 1 00000100:00100000:2.0:1789760758.800036:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507287424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760758.800113:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20930560, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:2.0:1789760758.800124:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800144:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760758.800149:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.800156:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800173:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.834549:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972430 is committed 00002000:00010000:2.0:1789760758.834794:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20930560 -> 20934656 00010000:00010000:2.0:1789760758.834816:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834832:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834843:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834852:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.834859:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834870:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834903:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.834912:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b87f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.835117:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507287424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35075us (35367us total) trans 4294972430 rc 0/0 00000100:00100000:2.0:1789760758.835130:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5140 00000100:00100000:2.0:1789760758.835142:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740049872382 time 740115324673 state 0x2 ref 0 00000100:00100000:2.0:1789760758.838449:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760758.838845:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287552 00000100:00100000:0.0:1789760758.838860:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287552 00000100:00100000:0.0:1789760758.838951:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740049872382 time 740115324673 state 0x0 ref 1 00000100:00100000:0.0:1789760758.838973:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740115585089 time 740115324673 state 0x1 ref 1 00000100:00100000:0.0:1789760758.839073:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740115585089 time 740154373927 state 0x0 ref 1 00000100:00100000:0.0:1789760758.839089:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760758.839154:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20934656, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:0.0:1789760758.839166:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.839186:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760758.839192:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.839199:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.839215:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.882404:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972431 is committed 00002000:00010000:0.0:1789760758.886201:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20934656 -> 20938752 00010000:00010000:0.0:1789760758.886227:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886242:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886252:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886260:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760758.886266:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886276:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886304:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.886311:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b87fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.886479:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47385us (48038us total) trans 4294972431 rc 0/0 00000100:00100000:0.0:1789760758.886491:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5141 00000100:00100000:0.0:1789760758.886503:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740115585089 time 740154373927 state 0x2 ref 0 00000100:00100000:1.0:1789760758.890326:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.890473:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287680 00000100:00100000:2.0:1789760758.890487:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287680 00000100:00100000:2.0:1789760758.890562:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740115585089 time 740154373927 state 0x0 ref 1 00000100:00100000:2.0:1789760758.890581:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740154634343 time 740154373927 state 0x1 ref 1 00000100:00100000:2.0:1789760758.890648:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740154634343 time 740205949905 state 0x0 ref 1 00000100:00100000:2.0:1789760758.890672:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507287680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760758.890790:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20938752, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:2.0:1789760758.890802:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.890831:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760758.890836:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.890844:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.890859:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.924133:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20938752 -> 20942848 00010000:00010000:1.0:1789760758.924157:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924173:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924183:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924191:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760758.924197:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924214:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924238:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.924247:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8803 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.924360:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507287680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33694us (34043us total) trans 4294972432 rc 0/0 00000100:00100000:1.0:1789760758.924370:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5142 00000100:00100000:1.0:1789760758.924382:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740154634343 time 740205949905 state 0x2 ref 0 00000001:00080000:1.0:1789760758.924955:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972432 is committed 00000100:00100000:0.0:1789760758.928484:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760758.928609:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287808 00000100:00100000:2.0:1789760758.928622:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287808 00000100:00100000:2.0:1789760758.928915:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740154634343 time 740205949905 state 0x0 ref 1 00000100:00100000:2.0:1789760758.928938:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740206210321 time 740205949905 state 0x1 ref 1 00000100:00100000:3.0:1789760758.930229:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740206210321 time 740245521464 state 0x0 ref 1 00000100:00100000:3.0:1789760758.930261:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507287808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760758.930333:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20942848, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:3.0:1789760758.930345:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.930366:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760758.930372:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.930380:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.930396:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.966103:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20942848 -> 20946944 00010000:00010000:2.0:1789760758.966128:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966143:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966153:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966167:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760758.966173:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966184:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966223:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.966281:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b880a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.966417:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507287808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36162us (37941us total) trans 4294972433 rc 0/0 00000100:00100000:2.0:1789760758.966430:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5143 00000100:00100000:2.0:1789760758.966442:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740206210321 time 740245521464 state 0x2 ref 0 00000001:00080000:1.0:1789760758.968872:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972433 is committed 00000100:00100000:3.0:1789760758.978676:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760758.981975:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507287936 00000100:00100000:0.0:1789760758.982000:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507287936 00000100:00100000:0.0:1789760758.982091:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740206210321 time 740245521464 state 0x0 ref 1 00000100:00100000:0.0:1789760758.982112:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740245781880 time 740245521464 state 0x1 ref 1 00000100:00100000:0.0:1789760758.982577:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740245781880 time 740297875364 state 0x0 ref 1 00000100:00100000:0.0:1789760758.982598:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507287936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760758.982680:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20946944, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:0.0:1789760758.982740:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.982766:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760758.982772:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.982780:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.982796:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.004183:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538280960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004356:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.004425:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbf8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538281472:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004457:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.004482:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227828a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538281984:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004504:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.004533:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789760759.004539:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538280576:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004570:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.004593:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538281088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004620:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.004639:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538281600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.004674:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005113:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22782b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538282112:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005153:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005193:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbfaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538280704:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005359:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005387:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538281216:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005413:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005432:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b45aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538281728:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005451:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005474:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760759.005481:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538280832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005504:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005519:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21fbfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538281344:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005537:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005552:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227828000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538281856:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.005570:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.005607:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280960 00000100:00100000:3.0:1789760759.005617:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280960 00000100:00100000:3.0:1789760759.005678:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280704 00000100:00100000:3.0:1789760759.005681:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280704 00000100:00100000:3.0:1789760759.006326:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281216 00000100:00100000:3.0:1789760759.006332:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281216 00000100:00100000:3.0:1789760759.006934:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281728 00000100:00100000:3.0:1789760759.006940:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281728 00000100:00100000:1.0:1789760759.007628:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281344 00000100:00100000:1.0:1789760759.007637:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281344 00000100:00080000:0.0:1789760759.008256:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760759.008264:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760759.008320:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281856 00000100:00100000:2.0:1789760759.008327:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281856 00000100:00100000:3.0:1789760759.008460:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 922 00000100:00100000:3.0:1789760759.008473:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538280960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.008601:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538280960:12345-0@lo:400:kworker.0 Request processed in 129us (4245us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760759.008609:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 922 00000100:00100000:3.0:1789760759.008630:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 923 00000100:00100000:3.0:1789760759.008638:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538280704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.008686:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538280704:12345-0@lo:400:kworker.0 Request processed in 49us (3460us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.008748:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 924 00000100:00100000:2.0:1789760759.008757:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538281216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760759.008800:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538281216:12345-0@lo:400:kworker.0 Request processed in 45us (3386us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.008807:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 924 00000100:00100000:2.0:1789760759.008823:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:2.0:1789760759.008830:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538281728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760759.008860:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538281728:12345-0@lo:400:kworker.0 Request processed in 31us (3408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.008867:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:2.0:1789760759.008878:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:2.0:1789760759.008883:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538281856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760759.008911:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538281856:12345-0@lo:400:kworker.0 Request processed in 27us (3339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.008917:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:2.0:1789760759.008941:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281088 00000100:00100000:2.0:1789760759.008945:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281088 00000100:00100000:2.0:1789760759.008979:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280832 00000100:00100000:2.0:1789760759.008983:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280832 00000100:00080000:0.0:1789760759.008988:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760759.009005:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:2.0:1789760759.009013:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538281088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.009079:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:0.0:1789760759.009092:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538280832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.009137:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 923 00000100:00100000:3.0:1789760759.009207:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23ebeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538282368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760759.009284:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538281088:12345-0@lo:400:kworker.0 Request processed in 270us (4661us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.009291:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760759.009416:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:1.0:1789760759.009425:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538281344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.009574:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538280832:12345-0@lo:400:kworker.0 Request processed in 486us (4067us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.009580:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:0.0:1789760759.009607:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281472 00000100:00100000:0.0:1789760759.009612:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281472 00000100:00100000:0.0:1789760759.009645:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281984 00000100:00100000:0.0:1789760759.009649:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281984 00000100:00100000:0.0:1789760759.009675:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538281600 00000100:00100000:0.0:1789760759.009678:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538281600 00000100:00100000:0.0:1789760759.009721:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282112 00000100:00100000:0.0:1789760759.009725:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282112 00000100:00100000:0.0:1789760759.009756:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789760759.009762:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538281472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.009801:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538281472:12345-0@lo:400:kworker.0 Request processed in 38us (5342us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.009809:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789760759.009823:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:0.0:1789760759.010006:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538281984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.010038:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538281984:12345-0@lo:400:kworker.0 Request processed in 35us (5533us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.010044:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:0.0:1789760759.010056:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:0.0:1789760759.010061:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538281600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.010088:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538281600:12345-0@lo:400:kworker.0 Request processed in 27us (5414us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.010092:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:0.0:1789760759.010102:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:0.0:1789760759.010108:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227a03050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538282112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.010135:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227a03050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538282112:12345-0@lo:400:kworker.0 Request processed in 28us (4982us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.010141:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00080000:0.0:1789760759.010434:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760759.010438:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760759.011402:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.011461:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760759.011476:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22353c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538282240:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760759.011524:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.011621:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538280832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.011639:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227828000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538281856:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.011652:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbfaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538280704:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.011680:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538281216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760759.012058:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282368 00000100:00100000:0.0:1789760759.012067:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282368 00000100:00100000:0.0:1789760759.012124:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282240 00000100:00100000:0.0:1789760759.012127:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282240 00000100:00100000:0.0:1789760759.012148:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:0.0:1789760759.012158:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538282368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760759.012207:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760759.012238:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:2.0:1789760759.012250:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538282240:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760759.012267:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760759.012328:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538282240:12345-0@lo:13:osp-pre-1-1.0 Request processed in 81us (801us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760759.012337:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:0.0:1789760759.012347:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538282368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 190us (949us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.012356:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:3.0:1789760759.012604:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b45aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538281728:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.012686:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22353c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538282240:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760759.013341:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23ebeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538282368:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760759.013387:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538281088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.013402:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538281600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.013413:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22782b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538282112:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.013426:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538280960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.013440:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbf8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538281472:0@lo:400:kworker.0 00000100:00100000:3.0:1789760759.013451:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227828a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538281984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.013566:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538281344:12345-0@lo:400:kworker.0 Request processed in 4139us (8024us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.013578:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000100:00100000:1.0:1789760759.013617:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538280576 00000100:00100000:1.0:1789760759.013623:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538280576 00000100:00100000:3.0:1789760759.013915:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538281344:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.014222:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1789760759.014316:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538280576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.014416:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538280576:12345-0@lo:400:kworker.0 Request processed in 106us (9844us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.014424:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:0.0:1789760759.014552:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21fbf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538280576:0@lo:400:kworker.0 00000001:00080000:3.0:1789760759.041620:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972434 is committed 00002000:00010000:1.0:1789760759.041945:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:1.0:1789760759.041965:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.041979:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.041989:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.041996:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.042002:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.042012:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.042037:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.042045:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8811 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.042183:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507287936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59582us (63524us total) trans 4294972434 rc 0/0 00000100:00100000:1.0:1789760759.042198:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5144 00000100:00100000:1.0:1789760759.042211:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740245781880 time 740297875364 state 0x2 ref 0 00000100:00100000:2.0:1789760759.046122:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.047982:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288064 00000100:00100000:1.0:1789760759.048001:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288064 00000100:00100000:1.0:1789760759.048108:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740245781880 time 740297875364 state 0x0 ref 1 00000100:00100000:1.0:1789760759.048127:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740298135780 time 740297875364 state 0x1 ref 1 00000100:00100000:1.0:1789760759.048163:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740298135780 time 740363466034 state 0x0 ref 1 00000100:00100000:1.0:1789760759.048178:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.048244:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20951040, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xa007 00010000:00010000:1.0:1789760759.048256:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.048277:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.048282:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.048290:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.048306:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760759.048345:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760759.106723:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972435 is committed 00002000:00010000:0.0:1789760759.107150:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20951040 -> 20955136 00002000:00010000:0.0:1789760759.107154:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:0.0:1789760759.107157:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:0.0:1789760759.107171:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107185:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107193:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107201:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.107206:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107216:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107236:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.107244:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8818 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.107551:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59371us (61439us total) trans 4294972435 rc 0/0 00000100:00100000:0.0:1789760759.107563:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5145 00000100:00100000:0.0:1789760759.107573:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740298135780 time 740363466034 state 0x2 ref 0 00000100:00100000:0.0:1789760759.118449:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760759.118799:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288192 00000100:00100000:2.0:1789760759.118809:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288192 00000100:00100000:2.0:1789760759.118865:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740298135780 time 740363466034 state 0x0 ref 1 00000100:00100000:2.0:1789760759.118880:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740363726450 time 740363466034 state 0x1 ref 1 00000100:00100000:2.0:1789760759.118913:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740363726450 time 740434216726 state 0x0 ref 1 00000100:00100000:2.0:1789760759.118924:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760759.118973:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20955136, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa007 00010000:00010000:2.0:1789760759.118982:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.118996:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760759.119001:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.119007:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.119021:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.149847:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972436 is committed 00002000:00010000:0.0:1789760759.149953:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20955136 -> 20959232 00010000:00010000:0.0:1789760759.149968:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.149981:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.149989:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.149996:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.150001:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.150010:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.150029:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.150042:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b881f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.150133:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31205us (31689us total) trans 4294972436 rc 0/0 00000100:00100000:0.0:1789760759.150143:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5146 00000100:00100000:0.0:1789760759.150151:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740363726450 time 740434216726 state 0x2 ref 0 00000100:00100000:3.0:1789760759.153315:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.153375:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288320 00000100:00100000:1.0:1789760759.153384:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288320 00000100:00100000:1.0:1789760759.153461:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740363726450 time 740434216726 state 0x0 ref 1 00000100:00100000:1.0:1789760759.153476:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740434477142 time 740434216726 state 0x1 ref 1 00000100:00100000:1.0:1789760759.153501:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740434477142 time 740468804324 state 0x0 ref 1 00000100:00100000:1.0:1789760759.153512:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.153559:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20959232, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa007 00010000:00010000:1.0:1789760759.153567:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.153582:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.153586:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.153593:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.153607:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.174946:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972437 is committed 00002000:00010000:1.0:1789760759.175053:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:1.0:1789760759.175069:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175089:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175098:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175105:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.175110:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175119:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175138:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.175147:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8826 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.175280:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21764us (21969us total) trans 4294972437 rc 0/0 00000100:00100000:1.0:1789760759.175290:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5147 00000100:00100000:1.0:1789760759.175301:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740434477142 time 740468804324 state 0x2 ref 0 00000100:00100000:1.0:1789760759.185897:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.186120:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288448 00000100:00100000:0.0:1789760759.186131:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288448 00000100:00100000:0.0:1789760759.186194:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740434477142 time 740468804324 state 0x0 ref 1 00000100:00100000:0.0:1789760759.186212:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740469064740 time 740468804324 state 0x1 ref 1 00000100:00100000:0.0:1789760759.186242:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740469064740 time 740501545252 state 0x0 ref 1 00000100:00100000:0.0:1789760759.186254:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507288448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.186301:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20963328, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa007 00010000:00010000:0.0:1789760759.186310:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.186326:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760759.186331:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.186337:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.186351:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.223065:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972438 is committed 00002000:00010000:1.0:1789760759.223242:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:1.0:1789760759.223258:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223271:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223280:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223288:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.223292:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223302:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223321:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.223329:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b882d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.223423:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507288448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37166us (37532us total) trans 4294972438 rc 0/0 00000100:00100000:1.0:1789760759.223433:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5148 00000100:00100000:1.0:1789760759.223443:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740469064740 time 740501545252 state 0x2 ref 0 00000100:00100000:0.0:1789760759.230271:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760759.230367:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288576 00000100:00100000:2.0:1789760759.230378:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288576 00000100:00100000:2.0:1789760759.230444:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740469064740 time 740501545252 state 0x0 ref 1 00000100:00100000:2.0:1789760759.230460:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740501805668 time 740501545252 state 0x1 ref 1 00000100:00100000:2.0:1789760759.230492:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740501805668 time 740545795025 state 0x0 ref 1 00000100:00100000:2.0:1789760759.230505:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760759.230562:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20967424, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa007 00010000:00010000:2.0:1789760759.230573:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.230589:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760759.230593:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.230600:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.230614:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.260310:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972439 is committed 00002000:00010000:0.0:1789760759.260411:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:0.0:1789760759.260427:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260441:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260450:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260458:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.260463:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260471:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260490:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.260497:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8834 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.260605:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507288576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30098us (30338us total) trans 4294972439 rc 0/0 00000100:00100000:0.0:1789760759.260615:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5149 00000100:00100000:0.0:1789760759.260629:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740501805668 time 740545795025 state 0x2 ref 0 00000100:00100000:3.0:1789760759.264622:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.264735:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288704 00000100:00100000:1.0:1789760759.264745:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288704 00000100:00100000:1.0:1789760759.264796:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740501805668 time 740545795025 state 0x0 ref 1 00000100:00100000:1.0:1789760759.264810:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740546055441 time 740545795025 state 0x1 ref 1 00000100:00100000:1.0:1789760759.264835:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740546055441 time 740580139010 state 0x0 ref 1 00000100:00100000:1.0:1789760759.264847:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507288704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.264894:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20971520, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa007 00010000:00010000:1.0:1789760759.264903:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.264932:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.264937:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.264943:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.264956:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.297623:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972440 is committed 00002000:00010000:0.0:1789760759.297783:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:0.0:1789760759.297789:0:8090:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:0.0:1789760759.297802:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297814:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297824:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297831:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.297836:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297845:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297864:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.297879:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b883b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.298010:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507288704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33160us (33392us total) trans 4294972440 rc 0/0 00000100:00100000:0.0:1789760759.298021:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5150 00000100:00100000:0.0:1789760759.298030:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740546055441 time 740580139010 state 0x2 ref 0 00000100:00100000:1.0:1789760759.301075:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.301507:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288832 00000100:00100000:0.0:1789760759.301517:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288832 00000100:00100000:0.0:1789760759.301576:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740546055441 time 740580139010 state 0x0 ref 1 00000100:00100000:0.0:1789760759.301592:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740580399426 time 740580139010 state 0x1 ref 1 00000100:00100000:0.0:1789760759.301622:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740580399426 time 740616925168 state 0x0 ref 1 00000100:00100000:0.0:1789760759.301634:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507288832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.301769:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20975616, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:0.0:1789760759.301780:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.301797:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760759.301802:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.301808:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.301829:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.362296:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20975616 -> 20979712 00010000:00010000:1.0:1789760759.362316:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362335:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362344:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362351:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.362355:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362364:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362382:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.362388:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8842 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.362533:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972441 is committed 00000100:00100000:1.0:1789760759.362597:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507288832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60960us (61526us total) trans 4294972441 rc 0/0 00000100:00100000:1.0:1789760759.362609:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5151 00000100:00100000:1.0:1789760759.362618:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740580399426 time 740616925168 state 0x2 ref 0 00000100:00100000:2.0:1789760759.368250:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.368392:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507288960 00000100:00100000:0.0:1789760759.368402:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507288960 00000100:00100000:0.0:1789760759.368462:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740580399426 time 740616925168 state 0x0 ref 1 00000100:00100000:0.0:1789760759.368477:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740617185584 time 740616925168 state 0x1 ref 1 00000100:00100000:0.0:1789760759.368509:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740617185584 time 740683812924 state 0x0 ref 1 00000100:00100000:0.0:1789760759.368521:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507288960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.368567:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20979712, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:0.0:1789760759.368576:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.368597:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760759.368602:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.368608:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.368621:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.407149:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972442 is committed 00002000:00010000:0.0:1789760759.407365:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20979712 -> 20983808 00010000:00010000:0.0:1789760759.407380:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407391:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407399:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407406:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.407411:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407418:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407436:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.407443:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8849 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.407533:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507288960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39008us (39288us total) trans 4294972442 rc 0/0 00000100:00100000:0.0:1789760759.407543:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5152 00000100:00100000:0.0:1789760759.407553:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740617185584 time 740683812924 state 0x2 ref 0 00000100:00100000:0.0:1789760759.418564:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.418660:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289088 00000100:00100000:1.0:1789760759.418822:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289088 00000100:00100000:1.0:1789760759.418877:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740617185584 time 740683812924 state 0x0 ref 1 00000100:00100000:1.0:1789760759.418891:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740684073340 time 740683812924 state 0x1 ref 1 00000100:00100000:1.0:1789760759.418958:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740684073340 time 740734260569 state 0x0 ref 1 00000100:00100000:1.0:1789760759.418972:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507289088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.419024:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20983808, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760759.419033:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.419048:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.419052:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.419058:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.419071:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.447788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972443 is committed 00002000:00010000:3.0:1789760759.447997:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:3.0:1789760759.448014:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448025:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448034:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448042:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760759.448046:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448055:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448073:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.448080:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8850 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.448177:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507289088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29203us (29619us total) trans 4294972443 rc 0/0 00000100:00100000:3.0:1789760759.448187:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5153 00000100:00100000:3.0:1789760759.448197:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740684073340 time 740734260569 state 0x2 ref 0 00000100:00100000:1.0:1789760759.463457:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.463919:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289216 00000100:00100000:0.0:1789760759.463930:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289216 00000100:00100000:1.0:1789760759.464618:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740684073340 time 740734260569 state 0x0 ref 1 00000100:00100000:1.0:1789760759.464636:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740734520985 time 740734260569 state 0x1 ref 1 00000100:00100000:1.0:1789760759.464812:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740734520985 time 740780113484 state 0x0 ref 1 00000100:00100000:1.0:1789760759.464829:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.464880:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20987904, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760759.464890:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.464907:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.464911:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.464919:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.464933:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.499187:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972444 is committed 00002000:00010000:3.0:1789760759.499297:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20987904 -> 20992000 00010000:00010000:3.0:1789760759.499314:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499327:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499341:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499348:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760759.499352:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499361:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499380:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.499387:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8857 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.499479:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34649us (36027us total) trans 4294972444 rc 0/0 00000100:00100000:3.0:1789760759.499490:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5154 00000100:00100000:3.0:1789760759.499499:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740734520985 time 740780113484 state 0x2 ref 0 00000100:00100000:0.0:1789760759.512322:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760759.513144:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289344 00000100:00100000:2.0:1789760759.513154:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289344 00000100:00100000:2.0:1789760759.513214:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740734520985 time 740780113484 state 0x0 ref 1 00000100:00100000:2.0:1789760759.513229:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740780373900 time 740780113484 state 0x1 ref 1 00000100:00100000:2.0:1789760759.513264:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740780373900 time 740828567489 state 0x0 ref 1 00000100:00100000:2.0:1789760759.513278:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760759.513330:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20992000, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:2.0:1789760759.513339:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.513355:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760759.513360:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.513366:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.513380:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.554571:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972445 is committed 00002000:00010000:1.0:1789760759.554779:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20992000 -> 20996096 00010000:00010000:1.0:1789760759.554795:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554808:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554817:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554825:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.554829:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554839:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554858:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.554866:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b885e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.555006:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41726us (42691us total) trans 4294972445 rc 0/0 00000100:00100000:1.0:1789760759.555017:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5155 00000100:00100000:1.0:1789760759.555027:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740780373900 time 740828567489 state 0x2 ref 0 00000100:00100000:0.0:1789760759.559649:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760759.559829:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289472 00000100:00100000:3.0:1789760759.559839:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289472 00000100:00100000:3.0:1789760759.559890:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740780373900 time 740828567489 state 0x0 ref 1 00000100:00100000:3.0:1789760759.559904:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740828827905 time 740828567489 state 0x1 ref 1 00000100:00100000:3.0:1789760759.559936:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740828827905 time 740875239803 state 0x0 ref 1 00000100:00100000:3.0:1789760759.559948:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760759.559996:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20996096, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:3.0:1789760759.560010:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.560025:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760759.560029:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.560036:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.560050:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.591795:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972446 is committed 00002000:00010000:1.0:1789760759.592089:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:1.0:1789760759.592105:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592117:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592126:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592133:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.592139:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592148:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592167:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.592174:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8865 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.592258:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32307us (32614us total) trans 4294972446 rc 0/0 00000100:00100000:1.0:1789760759.592269:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5156 00000100:00100000:1.0:1789760759.592278:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740828827905 time 740875239803 state 0x2 ref 0 00000100:00100000:2.0:1789760759.601276:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760759.602990:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289600 00000100:00100000:3.0:1789760759.603003:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289600 00000100:00100000:3.0:1789760759.603063:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740828827905 time 740875239803 state 0x0 ref 1 00000100:00100000:3.0:1789760759.603078:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740875500219 time 740875239803 state 0x1 ref 1 00000100:00100000:3.0:1789760759.603123:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740875500219 time 740918425984 state 0x0 ref 1 00000100:00100000:3.0:1789760759.603136:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760759.603186:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21000192, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:3.0:1789760759.603196:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.603210:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760759.603215:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.603220:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.603232:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.644290:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00002000:00010000:1.0:1789760759.644553:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21000192 -> 21004288 00010000:00010000:1.0:1789760759.644568:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644581:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644590:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644603:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.644607:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644616:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644635:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.644644:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b886c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.644789:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41649us (43516us total) trans 4294972447 rc 0/0 00000100:00100000:1.0:1789760759.644799:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5157 00000100:00100000:1.0:1789760759.644809:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740875500219 time 740918425984 state 0x2 ref 0 00000100:00100000:0.0:1789760759.647891:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.647938:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289728 00000100:00100000:1.0:1789760759.647947:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289728 00000100:00100000:1.0:1789760759.647989:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740875500219 time 740918425984 state 0x0 ref 1 00000100:00100000:1.0:1789760759.648001:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740918686400 time 740918425984 state 0x1 ref 1 00000100:00100000:1.0:1789760759.648032:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740918686400 time 740963335074 state 0x0 ref 1 00000100:00100000:1.0:1789760759.648047:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.648093:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21004288, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760759.648111:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.648127:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.648131:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.648138:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.648151:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.676461:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972448 is committed 00002000:00010000:0.0:1789760759.676838:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:0.0:1789760759.676854:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676866:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676875:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676883:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.676888:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676898:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676917:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.676931:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8873 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.677062:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29011us (29174us total) trans 4294972448 rc 0/0 00000100:00100000:0.0:1789760759.677073:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5158 00000100:00100000:0.0:1789760759.677083:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740918686400 time 740963335074 state 0x2 ref 0 00000100:00100000:0.0:1789760759.680979:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.681068:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289856 00000100:00100000:1.0:1789760759.681078:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289856 00000100:00100000:1.0:1789760759.681133:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740918686400 time 740963335074 state 0x0 ref 1 00000100:00100000:1.0:1789760759.681147:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740963595490 time 740963335074 state 0x1 ref 1 00000100:00100000:1.0:1789760759.681178:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740963595490 time 740996481658 state 0x0 ref 1 00000100:00100000:1.0:1789760759.681193:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507289856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.681244:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21008384, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760759.681254:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.681269:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760759.681274:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.681280:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.681300:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.708351:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760759.718614:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972449 is committed 00002000:00010000:1.0:1789760759.718843:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21008384 -> 21012480 00010000:00010000:1.0:1789760759.718863:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718879:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718889:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718898:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.718903:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718915:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718945:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.718955:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b887a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.719064:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507289856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37868us (38088us total) trans 4294972449 rc 0/0 00000100:00100000:1.0:1789760759.719076:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5159 00000100:00100000:1.0:1789760759.719087:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740963595490 time 740996481658 state 0x2 ref 0 00000100:00100000:2.0:1789760759.722477:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.722590:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507289984 00000100:00100000:1.0:1789760759.722600:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507289984 00000100:00100000:1.0:1789760759.722664:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740963595490 time 740996481658 state 0x0 ref 1 00000100:00100000:1.0:1789760759.722680:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740996742074 time 740996481658 state 0x1 ref 1 00000100:00100000:1.0:1789760759.722742:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740996742074 time 741038044540 state 0x0 ref 1 00000100:00100000:1.0:1789760759.722755:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507289984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760759.722803:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21012480, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760759.722812:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.722827:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760759.722832:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.722838:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.722851:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.760035:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:1.0:1789760759.760057:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760070:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760080:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760088:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.760094:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760117:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760140:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.760148:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8881 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.760254:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507289984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37496us (37782us total) trans 4294972450 rc 0/0 00000100:00100000:1.0:1789760759.760264:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5160 00000100:00100000:1.0:1789760759.760274:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740996742074 time 741038044540 state 0x2 ref 0 00000001:00080000:0.0:1789760759.761515:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00000100:00100000:0.0:1789760759.764973:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760759.765223:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290112 00000100:00100000:2.0:1789760759.765233:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290112 00000100:00100000:2.0:1789760759.765289:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 740996742074 time 741038044540 state 0x0 ref 1 00000100:00100000:2.0:1789760759.765305:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741038304956 time 741038044540 state 0x1 ref 1 00000100:00100000:2.0:1789760759.765379:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741038304956 time 741080680541 state 0x0 ref 1 00000100:00100000:2.0:1789760759.765393:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760759.765441:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21016576, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:2.0:1789760759.765452:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.765469:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760759.765473:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.765479:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.765493:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.796545:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00002000:00010000:2.0:1789760759.796641:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:2.0:1789760759.796665:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796677:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796685:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796822:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760759.796827:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796836:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796861:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.796868:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8888 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.796966:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31569us (31996us total) trans 4294972451 rc 0/0 00000100:00100000:2.0:1789760759.796975:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5161 00000100:00100000:2.0:1789760759.796984:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741038304956 time 741080680541 state 0x2 ref 0 00000100:00100000:2.0:1789760759.800555:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.800928:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290240 00000100:00100000:1.0:1789760759.800940:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290240 00000100:00100000:0.0:1789760759.801057:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741038304956 time 741080680541 state 0x0 ref 1 00000100:00100000:0.0:1789760759.801069:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741080940957 time 741080680541 state 0x1 ref 1 00000100:00100000:0.0:1789760759.801095:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741080940957 time 741116397880 state 0x0 ref 1 00000100:00100000:0.0:1789760759.801115:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.801173:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21020672, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:0.0:1789760759.801182:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.801197:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760759.801202:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.801209:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.801223:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.826129:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00002000:00010000:0.0:1789760759.826303:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:0.0:1789760759.826318:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826332:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826341:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826348:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760759.826353:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826364:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826384:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.826393:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b888f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.826487:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25369us (25937us total) trans 4294972452 rc 0/0 00000100:00100000:0.0:1789760759.826498:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5162 00000100:00100000:0.0:1789760759.826508:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741080940957 time 741116397880 state 0x2 ref 0 00000100:00100000:1.0:1789760759.832904:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760759.833083:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290368 00000100:00100000:3.0:1789760759.833092:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290368 00000100:00100000:3.0:1789760759.833145:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741080940957 time 741116397880 state 0x0 ref 1 00000100:00100000:3.0:1789760759.833160:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741116658296 time 741116397880 state 0x1 ref 1 00000100:00100000:3.0:1789760759.833186:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741116658296 time 741148490381 state 0x0 ref 1 00000100:00100000:3.0:1789760759.833196:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760759.833246:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21024768, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:3.0:1789760759.833255:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.833287:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760759.833291:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.833302:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.833315:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.871023:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21024768 -> 21028864 00010000:00010000:1.0:1789760759.871042:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871056:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871067:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871074:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.871079:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871088:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871107:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.871116:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8896 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.871209:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507290368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38007us (38308us total) trans 4294972453 rc 0/0 00000100:00100000:1.0:1789760759.871219:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5163 00000100:00100000:1.0:1789760759.871229:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741116658296 time 741148490381 state 0x2 ref 0 00000001:00080000:0.0:1789760759.871298:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00000100:00100000:2.0:1789760759.874941:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.874998:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290496 00000100:00100000:0.0:1789760759.875012:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290496 00000100:00100000:0.0:1789760759.875069:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741116658296 time 741148490381 state 0x0 ref 1 00000100:00100000:0.0:1789760759.875084:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741148750797 time 741148490381 state 0x1 ref 1 00000100:00100000:0.0:1789760759.875158:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741148750797 time 741190460310 state 0x0 ref 1 00000100:00100000:0.0:1789760759.875172:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.875222:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21028864, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:0.0:1789760759.875231:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.875247:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760759.875251:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.875257:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.875270:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.900335:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00002000:00010000:1.0:1789760759.900430:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21028864 -> 21032960 00010000:00010000:1.0:1789760759.900446:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900458:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900467:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900475:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.900480:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900488:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900510:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.900518:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b889d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.900607:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25431us (25670us total) trans 4294972454 rc 0/0 00000100:00100000:1.0:1789760759.900616:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5164 00000100:00100000:1.0:1789760759.900626:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741148750797 time 741190460310 state 0x2 ref 0 00000100:00100000:2.0:1789760759.904789:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760759.904895:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290624 00000100:00100000:0.0:1789760759.904905:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290624 00000100:00100000:0.0:1789760759.904958:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741148750797 time 741190460310 state 0x0 ref 1 00000100:00100000:0.0:1789760759.904971:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741190720726 time 741190460310 state 0x1 ref 1 00000100:00100000:0.0:1789760759.905001:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741190720726 time 741220304431 state 0x0 ref 1 00000100:00100000:0.0:1789760759.905012:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507290624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760759.905058:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21032960, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:0.0:1789760759.905066:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.905081:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760759.905085:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.905091:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.905115:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.936864:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:1.0:1789760759.936884:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936898:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936907:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936915:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760759.936920:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936935:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936956:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.936964:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b88a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.937055:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507290624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32040us (32273us total) trans 4294972455 rc 0/0 00000100:00100000:1.0:1789760759.937066:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5165 00000100:00100000:1.0:1789760759.937076:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741190720726 time 741220304431 state 0x2 ref 0 00000001:00080000:0.0:1789760759.937318:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00000100:00100000:2.0:1789760759.940557:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760759.940631:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290752 00000100:00100000:1.0:1789760759.940639:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290752 00000100:00100000:1.0:1789760759.940840:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741190720726 time 741220304431 state 0x0 ref 1 00000100:00100000:1.0:1789760759.940857:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741220564847 time 741220304431 state 0x1 ref 1 00000100:00100000:2.0:1789760759.940968:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741220564847 time 741256269511 state 0x0 ref 1 00000100:00100000:2.0:1789760759.940985:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760759.941035:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21037056, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:2.0:1789760759.941044:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.941059:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760759.941064:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.941070:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.941084:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.019316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:1.0:1789760760.019745:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21037056 -> 21041152 00010000:00010000:1.0:1789760760.019760:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019773:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019788:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019794:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.019799:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019807:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019827:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.019835:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b88ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.019981:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78995us (79425us total) trans 4294972456 rc 0/0 00000100:00100000:1.0:1789760760.019992:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5166 00000100:00100000:1.0:1789760760.020002:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741220564847 time 741256269511 state 0x2 ref 0 00000100:00100000:2.0:1789760760.027408:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.027502:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507290880 00000100:00100000:1.0:1789760760.027512:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507290880 00000100:00100000:1.0:1789760760.027564:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741220564847 time 741256269511 state 0x0 ref 1 00000100:00100000:1.0:1789760760.027580:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741256529927 time 741256269511 state 0x1 ref 1 00000100:00100000:1.0:1789760760.027613:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741256529927 time 741342916261 state 0x0 ref 1 00000100:00100000:1.0:1789760760.027626:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.027734:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21041152, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xa807 00010000:00010000:1.0:1789760760.027743:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.027760:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.027765:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.027771:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.027784:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.028320:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538282496:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760760.028374:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760760.028448:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282496 00000100:00100000:0.0:1789760760.028454:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282496 00000100:00100000:0.0:1789760760.028488:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1789760760.028496:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538282496:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760760.028511:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3837788160 avail 3805282304 00000100:00100000:0.0:1789760760.028552:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538282496:12345-0@lo:13:osp-pre-0-1.0 Request processed in 57us (177us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760760.028559:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1789760760.028620:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538282496:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760760.029026:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760760.029036:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538282624:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760760.029068:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760760.029110:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282624 00000100:00100000:1.0:1789760760.029114:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282624 00000100:00100000:1.0:1789760760.029139:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1789760760.029145:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538282624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760760.029158:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3837788160 avail 3805282304 00000100:00100000:1.0:1789760760.029191:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538282624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760760.029200:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1789760760.029230:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538282624:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760760.053294:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972457 is committed 00002000:00010000:1.0:1789760760.053300:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21041152 -> 21045248 00002000:00010000:1.0:1789760760.053304:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:1.0:1789760760.053306:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:1.0:1789760760.053318:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053333:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053430:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053436:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.053439:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053445:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053461:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.053466:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.053571:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507290880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25942us (26168us total) trans 4294972457 rc 0/0 00000100:00100000:1.0:1789760760.053860:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5167 00000100:00100000:1.0:1789760760.053870:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741256529927 time 741342916261 state 0x2 ref 0 00000100:00100000:2.0:1789760760.061603:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.061851:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291008 00000100:00100000:1.0:1789760760.061862:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291008 00000100:00100000:1.0:1789760760.061920:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741256529927 time 741342916261 state 0x0 ref 1 00000100:00100000:1.0:1789760760.061936:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741343176677 time 741342916261 state 0x1 ref 1 00000100:00100000:1.0:1789760760.061966:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741343176677 time 741377269072 state 0x0 ref 1 00000100:00100000:1.0:1789760760.061978:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507291008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.062027:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21045248, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.062036:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.062053:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.062057:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.062064:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.062078:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.089426:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:1.0:1789760760.089616:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:1.0:1789760760.089631:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089646:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089663:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089670:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.089675:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089817:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089838:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.089847:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.089963:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507291008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27983us (29279us total) trans 4294972458 rc 0/0 00000100:00100000:1.0:1789760760.089977:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5168 00000100:00100000:1.0:1789760760.089986:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741343176677 time 741377269072 state 0x2 ref 0 00000100:00100000:2.0:1789760760.095987:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.096923:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291136 00000100:00100000:1.0:1789760760.096934:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291136 00000100:00100000:1.0:1789760760.096986:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741343176677 time 741377269072 state 0x0 ref 1 00000100:00100000:1.0:1789760760.097001:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741377529488 time 741377269072 state 0x1 ref 1 00000100:00100000:1.0:1789760760.097029:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741377529488 time 741412333107 state 0x0 ref 1 00000100:00100000:1.0:1789760760.097041:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.097196:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21049344, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.097420:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.097438:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.097443:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.097450:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.097465:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.142201:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972459 is committed 00002000:00010000:0.0:1789760760.142455:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:0.0:1789760760.142472:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142484:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142493:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142500:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760760.142504:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142513:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142532:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.142540:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b88c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.142675:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45630us (46693us total) trans 4294972459 rc 0/0 00000100:00100000:0.0:1789760760.142734:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5169 00000100:00100000:0.0:1789760760.142744:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741377529488 time 741412333107 state 0x2 ref 0 00000100:00100000:2.0:1789760760.149623:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.149730:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291264 00000100:00100000:1.0:1789760760.149740:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291264 00000100:00100000:1.0:1789760760.149796:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741377529488 time 741412333107 state 0x0 ref 1 00000100:00100000:1.0:1789760760.149811:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741412593523 time 741412333107 state 0x1 ref 1 00000100:00100000:1.0:1789760760.149841:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741412593523 time 741465144179 state 0x0 ref 1 00000100:00100000:1.0:1789760760.149854:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507291264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.149900:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21053440, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.149909:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.149924:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.149929:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.149938:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.149950:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.196403:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972460 is committed 00002000:00010000:1.0:1789760760.196549:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21053440 -> 21057536 00010000:00010000:1.0:1789760760.196565:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196577:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196585:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196594:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.196599:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196608:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196627:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.196635:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.196885:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507291264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47028us (47267us total) trans 4294972460 rc 0/0 00000100:00100000:1.0:1789760760.196896:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5170 00000100:00100000:1.0:1789760760.196905:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741412593523 time 741465144179 state 0x2 ref 0 00000100:00100000:2.0:1789760760.205494:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.205665:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291392 00000100:00100000:1.0:1789760760.205675:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291392 00000100:00100000:1.0:1789760760.205766:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741412593523 time 741465144179 state 0x0 ref 1 00000100:00100000:1.0:1789760760.205782:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741465404595 time 741465144179 state 0x1 ref 1 00000100:00100000:1.0:1789760760.205813:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741465404595 time 741521115725 state 0x0 ref 1 00000100:00100000:1.0:1789760760.205829:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.205876:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21057536, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.205885:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.205900:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.205905:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.205911:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.205924:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.246289:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972461 is committed 00002000:00010000:1.0:1789760760.246438:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:1.0:1789760760.246452:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246465:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246474:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246480:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.246485:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246495:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246516:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246523:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.246616:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40784us (41127us total) trans 4294972461 rc 0/0 00000100:00100000:1.0:1789760760.246626:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5171 00000100:00100000:1.0:1789760760.246635:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741465404595 time 741521115725 state 0x2 ref 0 00000100:00100000:3.0:1789760760.250253:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760760.250347:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291520 00000100:00100000:0.0:1789760760.250357:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291520 00000100:00100000:0.0:1789760760.250423:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741465404595 time 741521115725 state 0x0 ref 1 00000100:00100000:0.0:1789760760.250439:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741521376141 time 741521115725 state 0x1 ref 1 00000100:00100000:0.0:1789760760.250470:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741521376141 time 741565772981 state 0x0 ref 1 00000100:00100000:0.0:1789760760.250482:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760760.250531:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21061632, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:0.0:1789760760.250540:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.250554:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760760.250559:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.250565:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.250579:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.281385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00002000:00010000:1.0:1789760760.281508:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:1.0:1789760760.281524:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281537:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281547:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281555:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.281560:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281569:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281593:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.281600:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b88d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.281838:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507291520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31352us (31590us total) trans 4294972462 rc 0/0 00000100:00100000:1.0:1789760760.281849:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5172 00000100:00100000:1.0:1789760760.281859:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741521376141 time 741565772981 state 0x2 ref 0 00000100:00100000:3.0:1789760760.286240:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.286302:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291648 00000100:00100000:1.0:1789760760.286312:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291648 00000100:00100000:1.0:1789760760.286364:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741521376141 time 741565772981 state 0x0 ref 1 00000100:00100000:1.0:1789760760.286386:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741566033397 time 741565772981 state 0x1 ref 1 00000100:00100000:1.0:1789760760.286408:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741566033397 time 741601711591 state 0x0 ref 1 00000100:00100000:1.0:1789760760.286418:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.286460:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21065728, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.286468:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.286482:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.286486:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.286492:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.286505:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.337568:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00002000:00010000:1.0:1789760760.337684:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:1.0:1789760760.337727:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337740:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337749:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337756:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.337761:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337770:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337794:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337800:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.337881:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51460us (51646us total) trans 4294972463 rc 0/0 00000100:00100000:1.0:1789760760.337890:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5173 00000100:00100000:1.0:1789760760.337898:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741566033397 time 741601711591 state 0x2 ref 0 00000100:00100000:2.0:1789760760.346049:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.346147:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291776 00000100:00100000:1.0:1789760760.346156:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291776 00000100:00100000:1.0:1789760760.346230:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741566033397 time 741601711591 state 0x0 ref 1 00000100:00100000:1.0:1789760760.346245:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741601972007 time 741601711591 state 0x1 ref 1 00000100:00100000:1.0:1789760760.346328:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741601972007 time 741661631486 state 0x0 ref 1 00000100:00100000:1.0:1789760760.346340:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.346390:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21069824, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.346399:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.346414:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.346419:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.346425:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.346439:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.414942:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00002000:00010000:1.0:1789760760.415061:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21069824 -> 21073920 00010000:00010000:1.0:1789760760.415076:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415089:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415098:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415105:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.415110:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415119:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415139:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.415149:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.415297:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68954us (69260us total) trans 4294972464 rc 0/0 00000100:00100000:1.0:1789760760.415309:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5174 00000100:00100000:1.0:1789760760.415319:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741601972007 time 741661631486 state 0x2 ref 0 00000100:00100000:0.0:1789760760.418994:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.419067:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507291904 00000100:00100000:1.0:1789760760.419077:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507291904 00000100:00100000:1.0:1789760760.419140:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741601972007 time 741661631486 state 0x0 ref 1 00000100:00100000:1.0:1789760760.419156:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741661891902 time 741661631486 state 0x1 ref 1 00000100:00100000:1.0:1789760760.419184:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741661891902 time 741734487048 state 0x0 ref 1 00000100:00100000:1.0:1789760760.419196:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.419250:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21073920, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.419260:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.419276:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.419292:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.419299:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.419314:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.452386:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972465 is committed 00002000:00010000:1.0:1789760760.452520:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:1.0:1789760760.452535:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452548:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452557:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452565:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.452569:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452579:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452599:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.452608:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.452753:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507291904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33554us (33765us total) trans 4294972465 rc 0/0 00000100:00100000:1.0:1789760760.452767:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5175 00000100:00100000:1.0:1789760760.452777:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741661891902 time 741734487048 state 0x2 ref 0 00000100:00100000:3.0:1789760760.456397:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.456484:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292032 00000100:00100000:1.0:1789760760.456496:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292032 00000100:00100000:1.0:1789760760.456551:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741661891902 time 741734487048 state 0x0 ref 1 00000100:00100000:1.0:1789760760.456566:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741734747464 time 741734487048 state 0x1 ref 1 00000100:00100000:1.0:1789760760.456602:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741734747464 time 741771905721 state 0x0 ref 1 00000100:00100000:1.0:1789760760.456615:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.456674:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21078016, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.456688:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.456734:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.456738:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.456745:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.456758:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.492926:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21078016 -> 21082112 00010000:00010000:2.0:1789760760.492945:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.492959:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.492968:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.492975:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.492981:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.492996:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.493015:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.493023:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b88f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.493110:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507292032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36492us (36718us total) trans 4294972466 rc 0/0 00000100:00100000:2.0:1789760760.493121:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5176 00000100:00100000:2.0:1789760760.493131:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741734747464 time 741771905721 state 0x2 ref 0 00000001:00080000:2.0:1789760760.493361:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972466 is committed 00000100:00100000:2.0:1789760760.496329:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.496401:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292160 00000100:00100000:1.0:1789760760.496411:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292160 00000100:00100000:1.0:1789760760.496465:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741734747464 time 741771905721 state 0x0 ref 1 00000100:00100000:1.0:1789760760.496480:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741772166137 time 741771905721 state 0x1 ref 1 00000100:00100000:1.0:1789760760.496511:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741772166137 time 741811814178 state 0x0 ref 1 00000100:00100000:1.0:1789760760.496524:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.496573:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21082112, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.496581:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.496595:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.496599:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.496606:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.496620:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.527628:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972467 is committed 00002000:00010000:0.0:1789760760.527749:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:0.0:1789760760.527765:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527777:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527786:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527794:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760760.527798:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527808:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527828:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.527836:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.527929:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31402us (31605us total) trans 4294972467 rc 0/0 00000100:00100000:0.0:1789760760.527939:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5177 00000100:00100000:0.0:1789760760.527949:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741772166137 time 741811814178 state 0x2 ref 0 00000100:00100000:0.0:1789760760.534609:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.534876:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292288 00000100:00100000:1.0:1789760760.534886:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292288 00000100:00100000:1.0:1789760760.534940:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741772166137 time 741811814178 state 0x0 ref 1 00000100:00100000:1.0:1789760760.534956:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741812074594 time 741811814178 state 0x1 ref 1 00000100:00100000:1.0:1789760760.535026:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741812074594 time 741850328764 state 0x0 ref 1 00000100:00100000:1.0:1789760760.535044:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.535155:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21086208, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.535165:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.535181:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760760.535186:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.535192:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.535206:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.540368:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760760.540390:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538282752:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760760.540454:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760760.540519:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282752 00000100:00100000:1.0:1789760760.540526:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282752 00000100:00100000:1.0:1789760760.540568:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760760.540576:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538282752:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760760.540612:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760760.540669:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538282752:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760760.540676:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760760.540745:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538282752:0@lo:41:osp-pre-1-0.0 00002000:00010000:2.0:1789760760.564262:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:2.0:1789760760.564282:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564296:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564305:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564318:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.564323:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564333:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564352:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.564362:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b88ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.564457:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507292288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29410us (29851us total) trans 4294972468 rc 0/0 00000100:00100000:2.0:1789760760.564467:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5178 00000100:00100000:2.0:1789760760.564478:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741812074594 time 741850328764 state 0x2 ref 0 00000001:00080000:0.0:1789760760.565577:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00000100:00100000:2.0:1789760760.568050:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760760.568190:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292416 00000100:00100000:3.0:1789760760.568200:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292416 00000100:00100000:3.0:1789760760.568253:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741812074594 time 741850328764 state 0x0 ref 1 00000100:00100000:3.0:1789760760.568269:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741850589180 time 741850328764 state 0x1 ref 1 00000100:00100000:3.0:1789760760.568299:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741850589180 time 741883601918 state 0x0 ref 1 00000100:00100000:3.0:1789760760.568312:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760760.568361:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21090304, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:3.0:1789760760.568371:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.568386:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760760.568391:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.568397:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.568412:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.598766:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:2.0:1789760760.598785:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598797:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598811:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598818:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.598823:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598833:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598856:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.598865:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8906 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.598966:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507292416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30651us (30920us total) trans 4294972469 rc 0/0 00000100:00100000:2.0:1789760760.598977:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5179 00000100:00100000:2.0:1789760760.598987:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741850589180 time 741883601918 state 0x2 ref 0 00000001:00080000:0.0:1789760760.599210:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972469 is committed 00000100:00100000:3.0:1789760760.605933:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760760.609062:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292544 00000100:00100000:0.0:1789760760.609078:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292544 00000100:00100000:0.0:1789760760.609138:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741850589180 time 741883601918 state 0x0 ref 1 00000100:00100000:0.0:1789760760.609153:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741883862334 time 741883601918 state 0x1 ref 1 00000100:00100000:0.0:1789760760.609234:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741883862334 time 741924536249 state 0x0 ref 1 00000100:00100000:0.0:1789760760.609249:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760760.609299:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21094400, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:0.0:1789760760.609308:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.609324:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760760.609329:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.609336:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.609350:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.688817:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972470 is committed 00002000:00010000:2.0:1789760760.688918:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21094400 -> 21098496 00010000:00010000:2.0:1789760760.688934:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.688952:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.688961:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.688968:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.688973:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.688983:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.689007:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.689014:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b890d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.689108:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 79858us (83181us total) trans 4294972470 rc 0/0 00000100:00100000:2.0:1789760760.689120:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5180 00000100:00100000:2.0:1789760760.689129:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741883862334 time 741924536249 state 0x2 ref 0 00000100:00100000:3.0:1789760760.698510:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760760.698614:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292672 00000100:00100000:2.0:1789760760.698624:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292672 00000100:00100000:2.0:1789760760.698685:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741883862334 time 741924536249 state 0x0 ref 1 00000100:00100000:2.0:1789760760.698729:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741924796665 time 741924536249 state 0x1 ref 1 00000100:00100000:2.0:1789760760.698757:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741924796665 time 742014060507 state 0x0 ref 1 00000100:00100000:2.0:1789760760.698770:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760760.698872:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21098496, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:0.0:1789760760.698883:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.698899:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760760.698903:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.698909:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.698923:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.733002:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760760.771877:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00002000:00010000:1.0:1789760760.772057:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:1.0:1789760760.772073:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772086:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772108:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772116:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.772121:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772130:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772151:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.772159:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8914 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.772290:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 73517us (73784us total) trans 4294972471 rc 0/0 00000100:00100000:1.0:1789760760.772306:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5181 00000100:00100000:1.0:1789760760.772316:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741924796665 time 742014060507 state 0x2 ref 0 00000100:00100000:1.0:1789760760.776623:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760760.776730:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292800 00000100:00100000:2.0:1789760760.776740:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292800 00000100:00100000:2.0:1789760760.776796:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 741924796665 time 742014060507 state 0x0 ref 1 00000100:00100000:2.0:1789760760.776812:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742014320923 time 742014060507 state 0x1 ref 1 00000100:00100000:2.0:1789760760.776946:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742014320923 time 742092247003 state 0x0 ref 1 00000100:00100000:2.0:1789760760.776961:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760760.777010:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21102592, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:2.0:1789760760.777020:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.777036:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760760.777042:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.777048:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.777067:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.827374:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00002000:00010000:1.0:1789760760.827534:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:1.0:1789760760.827550:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827562:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827571:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827579:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760760.827584:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827593:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827612:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.827619:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b891b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.828123:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507292800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51159us (51505us total) trans 4294972472 rc 0/0 00000100:00100000:1.0:1789760760.828135:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5182 00000100:00100000:1.0:1789760760.828144:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742014320923 time 742092247003 state 0x2 ref 0 00000100:00100000:3.0:1789760760.832018:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760760.832797:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507292928 00000100:00100000:2.0:1789760760.832808:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507292928 00000100:00100000:2.0:1789760760.832876:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742014320923 time 742092247003 state 0x0 ref 1 00000100:00100000:2.0:1789760760.832891:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742092507419 time 742092247003 state 0x1 ref 1 00000100:00100000:2.0:1789760760.832924:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742092507419 time 742148226656 state 0x0 ref 1 00000100:00100000:2.0:1789760760.832937:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760760.832981:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21106688, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:2.0:1789760760.832990:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.833005:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760760.833010:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.833017:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.833032:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.872899:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972473 is committed 00002000:00010000:2.0:1789760760.872909:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:2.0:1789760760.872924:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872938:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872947:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872955:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.872960:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872970:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872990:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.872998:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8922 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.873124:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507292928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40185us (41113us total) trans 4294972473 rc 0/0 00000100:00100000:2.0:1789760760.873139:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5183 00000100:00100000:2.0:1789760760.873149:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742092507419 time 742148226656 state 0x2 ref 0 00000100:00100000:3.0:1789760760.876559:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760760.877032:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293056 00000100:00100000:1.0:1789760760.877042:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293056 00000100:00100000:1.0:1789760760.877212:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742092507419 time 742148226656 state 0x0 ref 1 00000100:00100000:1.0:1789760760.877227:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742148487072 time 742148226656 state 0x1 ref 1 00000100:00100000:1.0:1789760760.877292:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742148487072 time 742192593641 state 0x0 ref 1 00000100:00100000:1.0:1789760760.877305:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760760.877352:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21110784, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:1.0:1789760760.877360:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.877375:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760760.877379:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.877385:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.877398:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.914385:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972474 is committed 00002000:00010000:2.0:1789760760.914541:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:2.0:1789760760.914556:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914574:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914584:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914591:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.914595:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914604:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914623:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.914630:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8929 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.914786:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37478us (38243us total) trans 4294972474 rc 0/0 00000100:00100000:2.0:1789760760.914796:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5184 00000100:00100000:2.0:1789760760.914806:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742148487072 time 742192593641 state 0x2 ref 0 00000100:00100000:1.0:1789760760.919419:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760760.919485:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293184 00000100:00100000:2.0:1789760760.919494:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293184 00000100:00100000:2.0:1789760760.919544:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742148487072 time 742192593641 state 0x0 ref 1 00000100:00100000:2.0:1789760760.919558:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742192854057 time 742192593641 state 0x1 ref 1 00000100:00100000:2.0:1789760760.919584:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742192854057 time 742234888392 state 0x0 ref 1 00000100:00100000:2.0:1789760760.919595:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760760.919638:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21114880, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:2.0:1789760760.919664:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.919685:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760760.919689:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.919732:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.919747:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.950488:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972475 is committed 00002000:00010000:2.0:1789760760.950611:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21114880 -> 21118976 00010000:00010000:2.0:1789760760.950625:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950639:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950658:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950666:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760760.950671:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950804:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950825:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950833:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8930 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.950952:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31354us (31537us total) trans 4294972475 rc 0/0 00000100:00100000:2.0:1789760760.950966:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5185 00000100:00100000:2.0:1789760760.950976:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742192854057 time 742234888392 state 0x2 ref 0 00000100:00100000:3.0:1789760760.955478:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760760.955561:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293312 00000100:00100000:2.0:1789760760.955570:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293312 00000100:00100000:2.0:1789760760.955619:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742192854057 time 742234888392 state 0x0 ref 1 00000100:00100000:2.0:1789760760.955634:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742235148808 time 742234888392 state 0x1 ref 1 00000100:00100000:2.0:1789760760.955681:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742235148808 time 742270982947 state 0x0 ref 1 00000100:00100000:2.0:1789760760.955889:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760760.955943:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21118976, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:2.0:1789760760.955953:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.955968:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760760.956046:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.956054:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.956071:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.016870:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:1.0:1789760761.017474:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21118976 -> 21123072 00010000:00010000:1.0:1789760761.017492:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017504:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017513:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017520:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760761.017525:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017534:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017554:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.017562:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8937 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.017674:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61978us (62199us total) trans 4294972476 rc 0/0 00000100:00100000:1.0:1789760761.017684:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5186 00000100:00100000:1.0:1789760761.017731:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742235148808 time 742270982947 state 0x2 ref 0 00000100:00100000:3.0:1789760761.023359:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760761.023580:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293440 00000100:00100000:2.0:1789760761.023591:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293440 00000100:00100000:2.0:1789760761.023777:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742235148808 time 742270982947 state 0x0 ref 1 00000100:00100000:2.0:1789760761.023793:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742271243363 time 742270982947 state 0x1 ref 1 00000100:00100000:2.0:1789760761.023814:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742271243363 time 742339118085 state 0x0 ref 1 00000100:00100000:2.0:1789760761.023826:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.023871:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21123072, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xa807 00010000:00010000:2.0:1789760761.023880:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.023896:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760761.023900:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.023907:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.023921:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760761.025865:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760761.052316:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760761.052339:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104735500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538282880:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760761.052416:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.052760:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538282880 00000100:00100000:0.0:1789760761.052768:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538282880 00000100:00100000:0.0:1789760761.052814:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1789760761.052824:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538282880:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760761.052865:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760761.052919:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538282880:12345-0@lo:41:osp-pre-0-1.0 Request processed in 93us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.052925:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1789760761.053066:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104735500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538282880:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760761.060460:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00002000:00010000:1.0:1789760761.060508:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21123072 -> 21127168 00002000:00010000:1.0:1789760761.060513:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:1.0:1789760761.060516:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:1.0:1789760761.060529:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060540:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060546:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060550:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760761.060554:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060561:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060576:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.060582:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b893e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.060669:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36839us (37314us total) trans 4294972477 rc 0/0 00000100:00100000:1.0:1789760761.060678:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5187 00000100:00100000:1.0:1789760761.060687:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742271243363 time 742339118085 state 0x2 ref 0 00000100:00100000:2.0:1789760761.063658:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760761.063783:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293568 00000100:00100000:1.0:1789760761.063793:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293568 00000100:00100000:1.0:1789760761.063847:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742271243363 time 742339118085 state 0x0 ref 1 00000100:00100000:1.0:1789760761.063862:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742339378501 time 742339118085 state 0x1 ref 1 00000100:00100000:1.0:1789760761.063886:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742339378501 time 742379190255 state 0x0 ref 1 00000100:00100000:1.0:1789760761.063898:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760761.063950:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21127168, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:1.0:1789760761.063958:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.063972:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760761.063977:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.063982:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.064000:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.085548:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00002000:00010000:1.0:1789760761.085584:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:1.0:1789760761.085600:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085613:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085622:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085629:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760761.085634:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085774:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085798:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.085806:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8945 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.085938:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507293568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22036us (22295us total) trans 4294972478 rc 0/0 00000100:00100000:1.0:1789760761.085948:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5188 00000100:00100000:1.0:1789760761.085957:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742339378501 time 742379190255 state 0x2 ref 0 00000100:00100000:2.0:1789760761.091884:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760761.091936:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293696 00000100:00100000:1.0:1789760761.091944:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293696 00000100:00100000:1.0:1789760761.091993:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742339378501 time 742379190255 state 0x0 ref 1 00000100:00100000:1.0:1789760761.092005:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742379450671 time 742379190255 state 0x1 ref 1 00000100:00100000:1.0:1789760761.092028:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742379450671 time 742407330704 state 0x0 ref 1 00000100:00100000:1.0:1789760761.092041:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760761.092089:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21131264, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:1.0:1789760761.092098:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.092114:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760761.092118:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.092125:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.092139:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.120186:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972479 is committed 00002000:00010000:1.0:1789760761.120295:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21131264 -> 21135360 00010000:00010000:1.0:1789760761.120311:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120324:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120333:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120340:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760761.120345:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120355:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120376:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.120384:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b894c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.120509:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507293696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28465us (28627us total) trans 4294972479 rc 0/0 00000100:00100000:1.0:1789760761.120523:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5189 00000100:00100000:1.0:1789760761.120532:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742379450671 time 742407330704 state 0x2 ref 0 00000100:00100000:3.0:1789760761.125275:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760761.127038:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293824 00000100:00100000:1.0:1789760761.127051:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293824 00000100:00100000:1.0:1789760761.127118:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742379450671 time 742407330704 state 0x0 ref 1 00000100:00100000:1.0:1789760761.127135:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742407591120 time 742407330704 state 0x1 ref 1 00000100:00100000:1.0:1789760761.127158:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742407591120 time 742442461899 state 0x0 ref 1 00000100:00100000:1.0:1789760761.127172:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507293824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760761.127231:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21135360, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:1.0:1789760761.127240:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.127257:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760761.127262:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.127269:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.127284:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.171294:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00002000:00010000:2.0:1789760761.171380:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21135360 -> 21139456 00010000:00010000:2.0:1789760761.171395:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171408:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171417:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171424:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.171429:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171439:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171459:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.171466:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8953 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.171601:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507293824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44426us (46330us total) trans 4294972480 rc 0/0 00000100:00100000:2.0:1789760761.171612:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5190 00000100:00100000:2.0:1789760761.171621:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742407591120 time 742442461899 state 0x2 ref 0 00000100:00100000:2.0:1789760761.180824:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.181058:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507293952 00000100:00100000:3.0:1789760761.181069:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507293952 00000100:00100000:3.0:1789760761.181125:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742407591120 time 742442461899 state 0x0 ref 1 00000100:00100000:3.0:1789760761.181141:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742442722315 time 742442461899 state 0x1 ref 1 00000100:00100000:3.0:1789760761.181173:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742442722315 time 742496476213 state 0x0 ref 1 00000100:00100000:3.0:1789760761.181189:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507293952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.181239:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21139456, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.181249:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.181266:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760761.181270:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.181276:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.181290:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.227593:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21139456 -> 21143552 00010000:00010000:3.0:1789760761.227615:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227629:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227651:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227659:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760761.227664:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227672:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227726:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.227736:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b895a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.227819:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507293952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46627us (47000us total) trans 4294972481 rc 0/0 00000100:00100000:3.0:1789760761.227829:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5191 00000100:00100000:3.0:1789760761.227838:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742442722315 time 742496476213 state 0x2 ref 0 00000001:00080000:3.0:1789760761.227874:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972481 is committed 00000100:00100000:0.0:1789760761.234272:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760761.234380:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294080 00000100:00100000:1.0:1789760761.234390:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294080 00000100:00100000:1.0:1789760761.234446:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742442722315 time 742496476213 state 0x0 ref 1 00000100:00100000:1.0:1789760761.234462:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742496736629 time 742496476213 state 0x1 ref 1 00000100:00100000:1.0:1789760761.234583:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742496736629 time 742549884630 state 0x0 ref 1 00000100:00100000:1.0:1789760761.234596:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760761.234672:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21143552, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:1.0:1789760761.234682:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.234726:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760761.234732:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.234739:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.234753:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.269042:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972482 is committed 00002000:00010000:1.0:1789760761.269259:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21143552 -> 21147648 00010000:00010000:1.0:1789760761.269274:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269285:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269358:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269368:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760761.269431:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269442:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269461:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.269468:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8961 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.269619:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35020us (35351us total) trans 4294972482 rc 0/0 00000100:00100000:1.0:1789760761.269630:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5192 00000100:00100000:1.0:1789760761.269649:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742496736629 time 742549884630 state 0x2 ref 0 00000100:00100000:0.0:1789760761.279288:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760761.279477:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294208 00000100:00100000:2.0:1789760761.279489:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294208 00000100:00100000:2.0:1789760761.279544:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742496736629 time 742549884630 state 0x0 ref 1 00000100:00100000:2.0:1789760761.279558:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742550145046 time 742549884630 state 0x1 ref 1 00000100:00100000:2.0:1789760761.279592:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742550145046 time 742594895449 state 0x0 ref 1 00000100:00100000:2.0:1789760761.279604:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.279667:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21147648, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.279676:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.279732:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760761.279738:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.279744:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.279760:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.326652:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00002000:00010000:2.0:1789760761.326807:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21147648 -> 21151744 00010000:00010000:2.0:1789760761.326823:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326836:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326846:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326854:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.326859:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326868:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326888:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.326895:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8968 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.326993:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47385us (47710us total) trans 4294972483 rc 0/0 00000100:00100000:2.0:1789760761.327005:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5193 00000100:00100000:2.0:1789760761.327016:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742550145046 time 742594895449 state 0x2 ref 0 00000100:00100000:2.0:1789760761.331608:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.331846:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294336 00000100:00100000:3.0:1789760761.331855:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294336 00000100:00100000:3.0:1789760761.331907:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742550145046 time 742594895449 state 0x0 ref 1 00000100:00100000:3.0:1789760761.331922:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742595155865 time 742594895449 state 0x1 ref 1 00000100:00100000:3.0:1789760761.331985:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742595155865 time 742647288525 state 0x0 ref 1 00000100:00100000:3.0:1789760761.331999:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507294336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.332048:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21151744, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.332058:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.332073:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760761.332077:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.332084:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.332108:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.364529:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:2.0:1789760761.364739:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21151744 -> 21155840 00010000:00010000:2.0:1789760761.364754:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364773:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364783:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364791:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.364796:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364805:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364824:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364832:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b896f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.364914:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507294336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32913us (33311us total) trans 4294972484 rc 0/0 00000100:00100000:2.0:1789760761.364924:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5194 00000100:00100000:2.0:1789760761.364934:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742595155865 time 742647288525 state 0x2 ref 0 00000100:00100000:0.0:1789760761.372161:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760761.374377:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294464 00000100:00100000:1.0:1789760761.374392:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294464 00000100:00100000:1.0:1789760761.374457:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742595155865 time 742647288525 state 0x0 ref 1 00000100:00100000:1.0:1789760761.374473:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742647548941 time 742647288525 state 0x1 ref 1 00000100:00100000:1.0:1789760761.374564:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742647548941 time 742689865601 state 0x0 ref 1 00000100:00100000:1.0:1789760761.374723:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507294464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760761.374773:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21155840, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:1.0:1789760761.374782:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.374799:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760761.374804:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.374810:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.374824:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.431538:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972485 is committed 00002000:00010000:2.0:1789760761.431744:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:2.0:1789760761.431760:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431773:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431781:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431788:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.431792:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431801:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431822:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.431830:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8976 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.431923:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507294464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57339us (59765us total) trans 4294972485 rc 0/0 00000100:00100000:2.0:1789760761.431933:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5195 00000100:00100000:2.0:1789760761.431942:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742647548941 time 742689865601 state 0x2 ref 0 00000100:00100000:0.0:1789760761.438289:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760761.438539:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294592 00000100:00100000:2.0:1789760761.438549:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294592 00000100:00100000:2.0:1789760761.438602:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742647548941 time 742689865601 state 0x0 ref 1 00000100:00100000:2.0:1789760761.438616:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742690126017 time 742689865601 state 0x1 ref 1 00000100:00100000:2.0:1789760761.438636:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742690126017 time 742753940182 state 0x0 ref 1 00000100:00100000:2.0:1789760761.438661:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507294592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.438741:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21159936, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.438750:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.438766:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760761.438771:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.438778:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.438792:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.474355:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972486 is committed 00002000:00010000:3.0:1789760761.475886:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:3.0:1789760761.475903:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475915:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475923:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475929:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760761.475934:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475943:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475964:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.475973:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b897d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.476113:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507294592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37449us (37829us total) trans 4294972486 rc 0/0 00000100:00100000:3.0:1789760761.476124:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5196 00000100:00100000:3.0:1789760761.476136:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742690126017 time 742753940182 state 0x2 ref 0 00000100:00100000:3.0:1789760761.481735:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760761.482285:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294720 00000100:00100000:2.0:1789760761.482298:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294720 00000100:00100000:2.0:1789760761.482355:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742690126017 time 742753940182 state 0x0 ref 1 00000100:00100000:2.0:1789760761.482370:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742754200598 time 742753940182 state 0x1 ref 1 00000100:00100000:2.0:1789760761.482406:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742754200598 time 742797709699 state 0x0 ref 1 00000100:00100000:2.0:1789760761.482419:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.482472:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21164032, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.482480:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.482497:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760761.482501:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.482508:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.482522:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.517761:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972487 is committed 00002000:00010000:2.0:1789760761.517855:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:2.0:1789760761.517869:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.517881:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.517891:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.517898:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.517903:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.517912:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.517930:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.518002:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8984 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.518124:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35703us (36429us total) trans 4294972487 rc 0/0 00000100:00100000:2.0:1789760761.518135:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5197 00000100:00100000:2.0:1789760761.518145:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742754200598 time 742797709699 state 0x2 ref 0 00000100:00100000:0.0:1789760761.523425:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760761.523796:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294848 00000100:00100000:2.0:1789760761.523805:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294848 00000100:00100000:2.0:1789760761.523860:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742754200598 time 742797709699 state 0x0 ref 1 00000100:00100000:2.0:1789760761.523874:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742797970115 time 742797709699 state 0x1 ref 1 00000100:00100000:2.0:1789760761.523929:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742797970115 time 742839232435 state 0x0 ref 1 00000100:00100000:2.0:1789760761.523947:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507294848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.523993:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21168128, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.524002:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.524017:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760761.524021:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.524028:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.524041:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.577530:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:3.0:1789760761.577550:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577564:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577572:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577579:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760761.577584:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577603:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577622:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.577629:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b898b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.577757:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507294848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53806us (54341us total) trans 4294972488 rc 0/0 00000100:00100000:3.0:1789760761.577767:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5198 00000100:00100000:3.0:1789760761.577777:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742797970115 time 742839232435 state 0x2 ref 0 00000001:00080000:0.0:1789760761.581353:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972488 is committed 00000100:00100000:0.0:1789760761.584813:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.584894:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507294976 00000100:00100000:3.0:1789760761.584906:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507294976 00000100:00100000:3.0:1789760761.584959:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742797970115 time 742839232435 state 0x0 ref 1 00000100:00100000:3.0:1789760761.584975:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742839492851 time 742839232435 state 0x1 ref 1 00000100:00100000:3.0:1789760761.585008:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742839492851 time 742900311187 state 0x0 ref 1 00000100:00100000:3.0:1789760761.585020:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.585071:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21172224, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.585080:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.585096:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760761.585101:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.585108:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.585121:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.626920:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972489 is committed 00002000:00010000:0.0:1789760761.627819:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:0.0:1789760761.627835:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627847:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627857:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627864:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760761.627869:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627879:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627904:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.627912:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8992 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.627999:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507294976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42976us (43192us total) trans 4294972489 rc 0/0 00000100:00100000:0.0:1789760761.628009:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5199 00000100:00100000:0.0:1789760761.628018:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742839492851 time 742900311187 state 0x2 ref 0 00000100:00100000:2.0:1789760761.643325:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.643425:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295104 00000100:00100000:3.0:1789760761.643437:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295104 00000100:00100000:3.0:1789760761.643491:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742839492851 time 742900311187 state 0x0 ref 1 00000100:00100000:3.0:1789760761.643517:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742900571603 time 742900311187 state 0x1 ref 1 00000100:00100000:3.0:1789760761.643544:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742900571603 time 742958847560 state 0x0 ref 1 00000100:00100000:3.0:1789760761.643556:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507295104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.643599:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21176320, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.643607:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.643622:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760761.643626:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.643632:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.643659:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.680343:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972490 is committed 00002000:00010000:2.0:1789760761.680447:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:2.0:1789760761.680463:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680476:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680486:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680493:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760761.680498:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680508:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680527:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.680535:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8999 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.680904:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507295104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37115us (37355us total) trans 4294972490 rc 0/0 00000100:00100000:2.0:1789760761.680916:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5200 00000100:00100000:2.0:1789760761.680926:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742900571603 time 742958847560 state 0x2 ref 0 00000100:00100000:2.0:1789760761.685551:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.685658:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295232 00000100:00100000:3.0:1789760761.685668:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295232 00000100:00100000:2.0:1789760761.686450:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742900571603 time 742958847560 state 0x0 ref 1 00000100:00100000:2.0:1789760761.686470:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742959107976 time 742958847560 state 0x1 ref 1 00000100:00100000:2.0:1789760761.686509:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742959107976 time 743001811756 state 0x0 ref 1 00000100:00100000:2.0:1789760761.686523:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507295232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.686574:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21180416, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.686583:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.686599:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760761.686604:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.686610:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.686627:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.762706:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760761.790522:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:0.0:1789760761.790542:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790556:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790565:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790572:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760761.790578:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790592:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790612:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.790620:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b89a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.790751:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507295232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 104226us (105204us total) trans 4294972491 rc 0/0 00000100:00100000:0.0:1789760761.790762:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5201 00000100:00100000:0.0:1789760761.790771:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742959107976 time 743001811756 state 0x2 ref 0 00000001:00080000:3.0:1789760761.796197:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972491 is committed 00000100:00100000:0.0:1789760761.798419:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.798509:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295360 00000100:00100000:3.0:1789760761.798520:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295360 00000100:00100000:3.0:1789760761.798572:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 742959107976 time 743001811756 state 0x0 ref 1 00000100:00100000:3.0:1789760761.798587:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743002072172 time 743001811756 state 0x1 ref 1 00000100:00100000:3.0:1789760761.798615:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743002072172 time 743113918112 state 0x0 ref 1 00000100:00100000:3.0:1789760761.798627:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507295360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.798724:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21184512, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.798733:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.798749:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760761.798754:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.798760:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.798774:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.851910:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21184512 -> 21188608 00010000:00010000:3.0:1789760761.851931:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.851945:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.851954:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.851960:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760761.851965:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.851973:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.852002:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.852010:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.852205:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507295360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53574us (53790us total) trans 4294972492 rc 0/0 00000100:00100000:3.0:1789760761.852218:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5202 00000100:00100000:3.0:1789760761.852227:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743002072172 time 743113918112 state 0x2 ref 0 00000001:00080000:0.0:1789760761.852299:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00000100:00100000:2.0:1789760761.857293:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760761.857383:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295488 00000100:00100000:0.0:1789760761.857393:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295488 00000100:00100000:0.0:1789760761.857521:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743002072172 time 743113918112 state 0x0 ref 1 00000100:00100000:0.0:1789760761.857537:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743114178528 time 743113918112 state 0x1 ref 1 00000100:00100000:3.0:1789760761.857908:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743114178528 time 743173208945 state 0x0 ref 1 00000100:00100000:3.0:1789760761.857920:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507295488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760761.857964:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21188608, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760761.857973:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.859241:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760761.859248:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.859256:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.859273:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.907004:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00002000:00010000:3.0:1789760761.907124:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21188608 -> 21192704 00010000:00010000:3.0:1789760761.907141:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907153:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907167:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907173:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760761.907178:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907186:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907206:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.907213:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.907311:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507295488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49388us (50022us total) trans 4294972493 rc 0/0 00000100:00100000:3.0:1789760761.907322:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5203 00000100:00100000:3.0:1789760761.907332:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743114178528 time 743173208945 state 0x2 ref 0 00000100:00100000:2.0:1789760761.911198:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760761.911292:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295616 00000100:00100000:0.0:1789760761.911302:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295616 00000100:00100000:0.0:1789760761.911357:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743114178528 time 743173208945 state 0x0 ref 1 00000100:00100000:0.0:1789760761.911371:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743173469361 time 743173208945 state 0x1 ref 1 00000100:00100000:0.0:1789760761.911401:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743173469361 time 743226705163 state 0x0 ref 1 00000100:00100000:0.0:1789760761.911413:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507295616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760761.911461:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21192704, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:0.0:1789760761.911469:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.911485:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760761.911490:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.911496:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.911508:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.966310:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00002000:00010000:0.0:1789760761.966677:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21192704 -> 21196800 00010000:00010000:0.0:1789760761.966722:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966735:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966745:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966752:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760761.966758:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966768:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966787:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.966794:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b89b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.966893:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507295616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55476us (55701us total) trans 4294972494 rc 0/0 00000100:00100000:0.0:1789760761.966903:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5204 00000100:00100000:0.0:1789760761.966914:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743173469361 time 743226705163 state 0x2 ref 0 00000100:00100000:0.0:1789760761.972389:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760761.972471:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295744 00000100:00100000:3.0:1789760761.972481:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295744 00000100:00100000:3.0:1789760761.972542:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743173469361 time 743226705163 state 0x0 ref 1 00000100:00100000:3.0:1789760761.972964:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743226965579 time 743226705163 state 0x1 ref 1 00000100:00100000:2.0:1789760761.973229:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743226965579 time 743288529458 state 0x0 ref 1 00000100:00100000:2.0:1789760761.973246:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507295744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760761.973295:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21196800, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:2.0:1789760761.973303:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.973319:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760761.973324:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.973331:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.973344:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.020883:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00002000:00010000:2.0:1789760762.021024:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21196800 -> 21200896 00010000:00010000:2.0:1789760762.021231:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021246:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021256:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021263:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760762.021268:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021277:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021296:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.021304:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b89bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.021431:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507295744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48184us (49046us total) trans 4294972495 rc 0/0 00000100:00100000:2.0:1789760762.021441:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5205 00000100:00100000:2.0:1789760762.021450:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743226965579 time 743288529458 state 0x2 ref 0 00000100:00100000:2.0:1789760762.025866:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.026101:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507295872 00000100:00100000:3.0:1789760762.026112:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507295872 00000100:00100000:3.0:1789760762.026170:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743226965579 time 743288529458 state 0x0 ref 1 00000100:00100000:3.0:1789760762.026185:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743288789874 time 743288529458 state 0x1 ref 1 00000100:00100000:3.0:1789760762.026214:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743288789874 time 743341518010 state 0x0 ref 1 00000100:00100000:3.0:1789760762.026227:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507295872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.026273:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21200896, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xa807 00010000:00010000:3.0:1789760762.026282:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.026298:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.026303:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.026309:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.026324:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.060055:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21200896 -> 21204992 00002000:00010000:3.0:1789760762.060064:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:3.0:1789760762.060067:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:3.0:1789760762.060254:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060269:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060279:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060286:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760762.060291:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060300:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060321:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.060328:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.060458:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507295872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34228us (34599us total) trans 4294972496 rc 0/0 00000100:00100000:3.0:1789760762.060470:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5206 00000100:00100000:3.0:1789760762.060480:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743288789874 time 743341518010 state 0x2 ref 0 00000001:00080000:0.0:1789760762.060539:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972496 is committed 00000100:00100000:1.0:1789760762.064979:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.065070:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296000 00000100:00100000:0.0:1789760762.065078:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296000 00000100:00100000:0.0:1789760762.065243:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743288789874 time 743341518010 state 0x0 ref 1 00000100:00100000:0.0:1789760762.065259:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743341778426 time 743341518010 state 0x1 ref 1 00000100:00100000:0.0:1789760762.065332:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743341778426 time 743380634357 state 0x0 ref 1 00000100:00100000:0.0:1789760762.065348:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.065395:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21204992, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.065404:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.065419:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.065424:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.065431:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.065446:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.096923:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21204992 -> 21209088 00010000:00010000:0.0:1789760762.096941:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.096954:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.096962:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.096969:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.096974:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.096983:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.097003:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.097010:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.097233:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507296000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31881us (32258us total) trans 4294972497 rc 0/0 00000100:00100000:0.0:1789760762.097243:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5207 00000100:00100000:0.0:1789760762.097253:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743341778426 time 743380634357 state 0x2 ref 0 00000001:00080000:0.0:1789760762.097289:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00000100:00100000:1.0:1789760762.100900:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760762.100980:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296128 00000100:00100000:2.0:1789760762.102405:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296128 00000100:00100000:2.0:1789760762.102476:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743341778426 time 743380634357 state 0x0 ref 1 00000100:00100000:2.0:1789760762.102491:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743380894773 time 743380634357 state 0x1 ref 1 00000100:00100000:2.0:1789760762.102568:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743380894773 time 743417870631 state 0x0 ref 1 00000100:00100000:2.0:1789760762.102582:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760762.102631:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21209088, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:2.0:1789760762.102649:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.102664:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760762.102676:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.102684:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.102712:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.132623:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972498 is committed 00002000:00010000:2.0:1789760762.132748:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21209088 -> 21213184 00010000:00010000:2.0:1789760762.132764:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132776:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132789:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132796:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760762.132801:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132810:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132828:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132836:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b89d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.132926:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30342us (32032us total) trans 4294972498 rc 0/0 00000100:00100000:2.0:1789760762.132936:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5208 00000100:00100000:2.0:1789760762.132946:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743380894773 time 743417870631 state 0x2 ref 0 00000100:00100000:2.0:1789760762.136837:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.136913:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296256 00000100:00100000:3.0:1789760762.136922:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296256 00000100:00100000:3.0:1789760762.136974:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743380894773 time 743417870631 state 0x0 ref 1 00000100:00100000:3.0:1789760762.136989:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743418131047 time 743417870631 state 0x1 ref 1 00000100:00100000:3.0:1789760762.137017:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743418131047 time 743452320632 state 0x0 ref 1 00000100:00100000:3.0:1789760762.137029:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.137076:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21213184, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.137101:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.137117:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.137122:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.137128:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.137141:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.208844:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:3.0:1789760762.208865:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208883:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208894:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208902:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760762.208906:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208915:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208935:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.208943:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.209054:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507296256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72020us (72219us total) trans 4294972499 rc 0/0 00000100:00100000:3.0:1789760762.209065:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5209 00000100:00100000:3.0:1789760762.209076:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743418131047 time 743452320632 state 0x2 ref 0 00000001:00080000:2.0:1789760762.209626:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972499 is committed 00000100:00100000:1.0:1789760762.212260:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.212500:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296384 00000100:00100000:3.0:1789760762.212511:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296384 00000100:00100000:3.0:1789760762.212626:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743418131047 time 743452320632 state 0x0 ref 1 00000100:00100000:3.0:1789760762.212663:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743452581048 time 743452320632 state 0x1 ref 1 00000100:00100000:3.0:1789760762.212717:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743452581048 time 743528020940 state 0x0 ref 1 00000100:00100000:3.0:1789760762.212730:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507296384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.212773:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21217280, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.212781:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.212797:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.212801:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.212807:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.212821:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.239257:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972500 is committed 00002000:00010000:1.0:1789760762.239352:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21217280 -> 21221376 00010000:00010000:1.0:1789760762.239368:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239381:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239392:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239401:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760762.239406:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239415:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239433:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.239442:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b89df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.239544:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507296384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26810us (27288us total) trans 4294972500 rc 0/0 00000100:00100000:1.0:1789760762.239556:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5210 00000100:00100000:1.0:1789760762.239566:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743452581048 time 743528020940 state 0x2 ref 0 00000100:00100000:1.0:1789760762.246863:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.246976:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296512 00000100:00100000:3.0:1789760762.246985:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296512 00000100:00100000:3.0:1789760762.247046:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743452581048 time 743528020940 state 0x0 ref 1 00000100:00100000:3.0:1789760762.247061:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743528281356 time 743528020940 state 0x1 ref 1 00000100:00100000:3.0:1789760762.247157:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743528281356 time 743562459655 state 0x0 ref 1 00000100:00100000:3.0:1789760762.247173:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.247225:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21221376, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.247234:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.247251:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.247255:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.247262:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.247277:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.330003:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00002000:00010000:0.0:1789760762.330126:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21221376 -> 21225472 00010000:00010000:0.0:1789760762.330142:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330154:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330163:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330174:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.330179:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330188:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330207:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.330215:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b89e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.330313:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 83137us (86647us total) trans 4294972501 rc 0/0 00000100:00100000:0.0:1789760762.330322:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5211 00000100:00100000:0.0:1789760762.330332:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743528281356 time 743562459655 state 0x2 ref 0 00000100:00100000:2.0:1789760762.333978:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.334069:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296640 00000100:00100000:3.0:1789760762.334173:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296640 00000100:00100000:3.0:1789760762.334227:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743528281356 time 743562459655 state 0x0 ref 1 00000100:00100000:3.0:1789760762.334244:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743562720071 time 743562459655 state 0x1 ref 1 00000100:00100000:3.0:1789760762.334301:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743562720071 time 743649604163 state 0x0 ref 1 00000100:00100000:3.0:1789760762.334314:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.334358:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21225472, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.334367:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.334387:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.334392:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.334397:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.334411:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.374959:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972502 is committed 00002000:00010000:2.0:1789760762.375100:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:2.0:1789760762.375116:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375128:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375137:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375144:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760762.375148:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375157:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375176:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.375184:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b89ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.375277:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507296640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40961us (41304us total) trans 4294972502 rc 0/0 00000100:00100000:2.0:1789760762.375286:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5212 00000100:00100000:2.0:1789760762.375297:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743562720071 time 743649604163 state 0x2 ref 0 00000100:00100000:2.0:1789760762.379013:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760762.379104:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296768 00000100:00100000:1.0:1789760762.379114:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296768 00000100:00100000:1.0:1789760762.379166:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743562720071 time 743649604163 state 0x0 ref 1 00000100:00100000:1.0:1789760762.379182:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743649864579 time 743649604163 state 0x1 ref 1 00000100:00100000:1.0:1789760762.379211:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743649864579 time 743694514554 state 0x0 ref 1 00000100:00100000:1.0:1789760762.379224:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21facc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507296768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760762.379269:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21229568, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:1.0:1789760762.379278:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.379293:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760762.379298:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.379304:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.379317:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.412660:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972503 is committed 00002000:00010000:0.0:1789760762.412892:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:0.0:1789760762.412909:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412922:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412936:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412943:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.412948:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412957:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412977:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.412984:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b89f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.413074:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21facc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507296768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33846us (34066us total) trans 4294972503 rc 0/0 00000100:00100000:0.0:1789760762.413154:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5213 00000100:00100000:0.0:1789760762.413165:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743649864579 time 743694514554 state 0x2 ref 0 00000100:00100000:3.0:1789760762.416896:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.417218:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507296896 00000100:00100000:0.0:1789760762.417229:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507296896 00000100:00100000:0.0:1789760762.417288:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743649864579 time 743694514554 state 0x0 ref 1 00000100:00100000:0.0:1789760762.417304:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743694774970 time 743694514554 state 0x1 ref 1 00000100:00100000:0.0:1789760762.417338:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743694774970 time 743732640845 state 0x0 ref 1 00000100:00100000:0.0:1789760762.417349:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507296896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.417395:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21233664, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.417403:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.417418:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.417423:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.417429:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.417443:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.447360:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21233664 -> 21237760 00010000:00010000:0.0:1789760762.447380:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447392:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447401:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447408:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.447414:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447423:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447441:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.447448:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b89fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.447537:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507296896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30183us (30646us total) trans 4294972504 rc 0/0 00000100:00100000:0.0:1789760762.447547:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5214 00000100:00100000:0.0:1789760762.447557:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743694774970 time 743732640845 state 0x2 ref 0 00000100:00100000:3.0:1789760762.450441:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.450530:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297024 00000100:00100000:0.0:1789760762.450540:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297024 00000100:00100000:0.0:1789760762.450592:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743694774970 time 743732640845 state 0x0 ref 1 00000100:00100000:0.0:1789760762.450608:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743732901261 time 743732640845 state 0x1 ref 1 00000100:00100000:0.0:1789760762.450647:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743732901261 time 743765949936 state 0x0 ref 1 00000100:00100000:0.0:1789760762.450659:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507297024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.450736:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21237760, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.450746:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.450762:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.450767:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.450773:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.450789:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.453360:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00000001:00080000:1.0:1789760762.482340:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00002000:00010000:3.0:1789760762.482424:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21237760 -> 21241856 00010000:00010000:3.0:1789760762.482442:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482455:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482464:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482471:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760762.482476:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482491:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482511:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.482519:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8a02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.482652:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507297024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31989us (32215us total) trans 4294972505 rc 0/0 00000100:00100000:3.0:1789760762.482662:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5215 00000100:00100000:3.0:1789760762.482682:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743732901261 time 743765949936 state 0x2 ref 0 00000100:00100000:3.0:1789760762.486494:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.486604:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297152 00000100:00100000:0.0:1789760762.486614:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297152 00000100:00100000:0.0:1789760762.488282:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743732901261 time 743765949936 state 0x0 ref 1 00000100:00100000:0.0:1789760762.488303:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743766210352 time 743765949936 state 0x1 ref 1 00000100:00100000:0.0:1789760762.488343:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743766210352 time 743803646228 state 0x0 ref 1 00000100:00100000:0.0:1789760762.488361:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.488411:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21241856, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.488420:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.488436:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.488440:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.488446:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.488462:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.518414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00002000:00010000:0.0:1789760762.518507:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:0.0:1789760762.518521:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518534:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518544:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518551:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.518556:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518566:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518585:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.518594:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.518798:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30439us (32308us total) trans 4294972506 rc 0/0 00000100:00100000:0.0:1789760762.518808:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5216 00000100:00100000:0.0:1789760762.518818:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743766210352 time 743803646228 state 0x2 ref 0 00000100:00100000:2.0:1789760762.521453:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.521515:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297280 00000100:00100000:3.0:1789760762.521525:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297280 00000100:00100000:3.0:1789760762.521576:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743766210352 time 743803646228 state 0x0 ref 1 00000100:00100000:3.0:1789760762.521592:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743803906644 time 743803646228 state 0x1 ref 1 00000100:00100000:3.0:1789760762.521617:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743803906644 time 743836920698 state 0x0 ref 1 00000100:00100000:3.0:1789760762.521640:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507297280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.521715:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21245952, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.521724:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.521739:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.521742:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.521747:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.521760:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.548921:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00002000:00010000:0.0:1789760762.549296:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:0.0:1789760762.549312:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549330:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549340:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549347:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.549352:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549369:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549385:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.549394:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8a10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.549472:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507297280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27839us (28023us total) trans 4294972507 rc 0/0 00000100:00100000:0.0:1789760762.549481:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5217 00000100:00100000:0.0:1789760762.549490:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743803906644 time 743836920698 state 0x2 ref 0 00000100:00100000:0.0:1789760762.553992:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.554113:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297408 00000100:00100000:3.0:1789760762.554123:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297408 00000100:00100000:3.0:1789760762.554176:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743803906644 time 743836920698 state 0x0 ref 1 00000100:00100000:3.0:1789760762.554191:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743837181114 time 743836920698 state 0x1 ref 1 00000100:00100000:3.0:1789760762.554262:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743837181114 time 743869564787 state 0x0 ref 1 00000100:00100000:3.0:1789760762.554274:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.554324:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21250048, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.554333:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.554350:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.554354:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.554360:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.554374:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.589980:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00002000:00010000:2.0:1789760762.590048:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:2.0:1789760762.590063:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590077:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590086:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590094:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760762.590098:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590106:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590125:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.590133:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8a17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.590224:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35946us (36236us total) trans 4294972508 rc 0/0 00000100:00100000:2.0:1789760762.590234:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5218 00000100:00100000:2.0:1789760762.590243:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743837181114 time 743869564787 state 0x2 ref 0 00000100:00100000:3.0:1789760762.594995:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.595323:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297536 00000100:00100000:0.0:1789760762.595334:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297536 00000100:00100000:0.0:1789760762.595462:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743837181114 time 743869564787 state 0x0 ref 1 00000100:00100000:0.0:1789760762.595477:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743869825203 time 743869564787 state 0x1 ref 1 00000100:00100000:0.0:1789760762.595506:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743869825203 time 743910808629 state 0x0 ref 1 00000100:00100000:0.0:1789760762.595519:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507297536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.595567:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21254144, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.595576:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.595591:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.595595:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.595602:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.595616:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.622922:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00002000:00010000:0.0:1789760762.623007:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:0.0:1789760762.623030:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623043:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623053:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623060:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.623065:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623176:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623196:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.623204:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.623290:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507297536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27769us (28299us total) trans 4294972509 rc 0/0 00000100:00100000:0.0:1789760762.623299:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5219 00000100:00100000:0.0:1789760762.623308:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743869825203 time 743910808629 state 0x2 ref 0 00000100:00100000:3.0:1789760762.626618:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.626725:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297664 00000100:00100000:0.0:1789760762.626735:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297664 00000100:00100000:0.0:1789760762.626789:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743869825203 time 743910808629 state 0x0 ref 1 00000100:00100000:0.0:1789760762.626804:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743911069045 time 743910808629 state 0x1 ref 1 00000100:00100000:0.0:1789760762.626835:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743911069045 time 743942137766 state 0x0 ref 1 00000100:00100000:0.0:1789760762.626848:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507297664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.626899:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21258240, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.626908:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.626924:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760762.626928:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.626934:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.626948:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.667349:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21258240 -> 21262336 00010000:00010000:0.0:1789760762.667380:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667394:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667404:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667411:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.667416:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667424:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667445:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.667452:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.667574:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507297664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40720us (40958us total) trans 4294972510 rc 0/0 00000100:00100000:0.0:1789760762.667590:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5220 00000100:00100000:0.0:1789760762.667604:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743911069045 time 743942137766 state 0x2 ref 0 00000001:00080000:3.0:1789760762.667868:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00000100:00100000:2.0:1789760762.672230:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.672338:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297792 00000100:00100000:3.0:1789760762.672349:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297792 00000100:00100000:3.0:1789760762.672462:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743911069045 time 743942137766 state 0x0 ref 1 00000100:00100000:3.0:1789760762.672477:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743942398182 time 743942137766 state 0x1 ref 1 00000100:00100000:3.0:1789760762.672597:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743942398182 time 743987898676 state 0x0 ref 1 00000100:00100000:3.0:1789760762.672611:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507297792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.672911:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21262336, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.672921:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.672937:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760762.672942:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.672947:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.672967:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.713372:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00002000:00010000:0.0:1789760762.713565:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:0.0:1789760762.713582:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713597:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713607:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713614:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.713619:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713637:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713662:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.713679:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8a2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.713816:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507297792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41203us (41592us total) trans 4294972511 rc 0/0 00000100:00100000:0.0:1789760762.713826:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5221 00000100:00100000:0.0:1789760762.713836:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743942398182 time 743987898676 state 0x2 ref 0 00000100:00100000:1.0:1789760762.717544:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.717861:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507297920 00000100:00100000:0.0:1789760762.717870:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507297920 00000100:00100000:0.0:1789760762.717923:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743942398182 time 743987898676 state 0x0 ref 1 00000100:00100000:0.0:1789760762.717937:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743988159092 time 743987898676 state 0x1 ref 1 00000100:00100000:0.0:1789760762.718010:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743988159092 time 744033312568 state 0x0 ref 1 00000100:00100000:0.0:1789760762.718032:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.718081:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21266432, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.718089:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.718105:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760762.718109:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.718116:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.718130:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.753493:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00002000:00010000:0.0:1789760762.753613:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21266432 -> 21270528 00010000:00010000:0.0:1789760762.753640:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753654:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753803:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753814:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.753819:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753828:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753853:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.753862:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8a33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.753972:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507297920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35946us (36434us total) trans 4294972512 rc 0/0 00000100:00100000:0.0:1789760762.753985:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5222 00000100:00100000:0.0:1789760762.753995:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743988159092 time 744033312568 state 0x2 ref 0 00000100:00100000:3.0:1789760762.759418:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.759473:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298048 00000100:00100000:0.0:1789760762.759483:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298048 00000100:00100000:0.0:1789760762.759539:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 743988159092 time 744033312568 state 0x0 ref 1 00000100:00100000:0.0:1789760762.759554:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744033572984 time 744033312568 state 0x1 ref 1 00000100:00100000:0.0:1789760762.759586:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744033572984 time 744074889595 state 0x0 ref 1 00000100:00100000:0.0:1789760762.759598:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507298048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.759657:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21270528, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.759675:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.759690:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760762.759727:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.759734:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.759752:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.779979:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760762.793655:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00002000:00010000:0.0:1789760762.794073:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21270528 -> 21274624 00010000:00010000:0.0:1789760762.794088:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794100:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794108:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794115:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.794119:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794128:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794149:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.794157:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.794255:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507298048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34654us (34842us total) trans 4294972513 rc 0/0 00000100:00100000:0.0:1789760762.794266:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5223 00000100:00100000:0.0:1789760762.794275:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744033572984 time 744074889595 state 0x2 ref 0 00000100:00100000:2.0:1789760762.799382:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.799520:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298176 00000100:00100000:0.0:1789760762.799530:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298176 00000100:00100000:0.0:1789760762.799579:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744033572984 time 744074889595 state 0x0 ref 1 00000100:00100000:0.0:1789760762.799594:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744075150011 time 744074889595 state 0x1 ref 1 00000100:00100000:0.0:1789760762.799670:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744075150011 time 744114965782 state 0x0 ref 1 00000100:00100000:0.0:1789760762.799684:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507298176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.799775:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21274624, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.799783:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.799798:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760762.799802:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.799808:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.799821:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.829741:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:0.0:1789760762.829969:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21274624 -> 21278720 00010000:00010000:0.0:1789760762.829985:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.829998:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.830012:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.830020:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.830025:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.830033:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.830055:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.830063:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.830156:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507298176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30470us (30777us total) trans 4294972514 rc 0/0 00000100:00100000:0.0:1789760762.830167:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5224 00000100:00100000:0.0:1789760762.830177:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744075150011 time 744114965782 state 0x2 ref 0 00000100:00100000:1.0:1789760762.844567:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.845254:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298304 00000100:00100000:0.0:1789760762.845266:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298304 00000100:00100000:0.0:1789760762.845321:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744075150011 time 744114965782 state 0x0 ref 1 00000100:00100000:0.0:1789760762.845337:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744115226198 time 744114965782 state 0x1 ref 1 00000100:00100000:0.0:1789760762.845369:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744115226198 time 744160672267 state 0x0 ref 1 00000100:00100000:0.0:1789760762.845382:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.845433:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21278720, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.845442:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.845468:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760762.845472:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.845478:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.845491:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.890424:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00002000:00010000:0.0:1789760762.890596:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21278720 -> 21282816 00010000:00010000:0.0:1789760762.890612:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890626:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890644:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890651:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.890656:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890792:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890815:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.890822:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.890952:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45567us (46388us total) trans 4294972515 rc 0/0 00000100:00100000:0.0:1789760762.890962:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5225 00000100:00100000:0.0:1789760762.890972:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744115226198 time 744160672267 state 0x2 ref 0 00000100:00100000:3.0:1789760762.900598:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760762.900756:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298432 00000100:00100000:0.0:1789760762.900767:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298432 00000100:00100000:0.0:1789760762.900823:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744115226198 time 744160672267 state 0x0 ref 1 00000100:00100000:0.0:1789760762.900845:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744160932683 time 744160672267 state 0x1 ref 1 00000100:00100000:0.0:1789760762.900878:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744160932683 time 744216181455 state 0x0 ref 1 00000100:00100000:0.0:1789760762.900891:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760762.900943:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21282816, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:0.0:1789760762.900952:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.900968:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760762.900973:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.900979:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.900992:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.948826:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00002000:00010000:0.0:1789760762.949408:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21282816 -> 21286912 00010000:00010000:0.0:1789760762.949422:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949434:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949604:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949800:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760762.949808:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949819:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949845:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.949852:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8a4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.949985:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49092us (49393us total) trans 4294972516 rc 0/0 00000100:00100000:0.0:1789760762.949994:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5226 00000100:00100000:0.0:1789760762.950003:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744160932683 time 744216181455 state 0x2 ref 0 00000100:00100000:0.0:1789760762.954559:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760762.954662:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298560 00000100:00100000:3.0:1789760762.954683:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298560 00000100:00100000:3.0:1789760762.954924:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744160932683 time 744216181455 state 0x0 ref 1 00000100:00100000:3.0:1789760762.954940:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744216441871 time 744216181455 state 0x1 ref 1 00000100:00100000:3.0:1789760762.955068:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744216441871 time 744270275182 state 0x0 ref 1 00000100:00100000:3.0:1789760762.955090:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760762.955135:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21286912, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:3.0:1789760762.955145:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.955160:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760762.955164:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.955170:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.955186:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.011895:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:3.0:1789760763.011917:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011931:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011940:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011948:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760763.011954:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011963:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011983:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.011991:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.012162:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507298560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57070us (57607us total) trans 4294972517 rc 0/0 00000100:00100000:3.0:1789760763.012173:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5227 00000100:00100000:3.0:1789760763.012182:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744216441871 time 744270275182 state 0x2 ref 0 00000001:00080000:3.0:1789760763.012220:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00000100:00100000:0.0:1789760763.016316:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.016412:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298688 00000100:00100000:1.0:1789760763.016422:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298688 00000100:00100000:1.0:1789760763.016494:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744216441871 time 744270275182 state 0x0 ref 1 00000100:00100000:1.0:1789760763.016510:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744270535598 time 744270275182 state 0x1 ref 1 00000100:00100000:1.0:1789760763.016541:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744270535598 time 744331843972 state 0x0 ref 1 00000100:00100000:1.0:1789760763.016554:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507298688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.016600:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21291008, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xa807 00010000:00010000:1.0:1789760763.016609:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.016625:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.016655:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.016668:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.016683:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760763.017319:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760763.104690:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972518 is committed 00002000:00010000:1.0:1789760763.104968:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21291008 -> 21295104 00002000:00010000:1.0:1789760763.104973:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:1.0:1789760763.104975:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:1.0:1789760763.104988:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105000:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105010:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105017:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760763.105191:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105204:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105225:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.105233:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.105323:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507298688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 88767us (89012us total) trans 4294972518 rc 0/0 00000100:00100000:1.0:1789760763.105333:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5228 00000100:00100000:1.0:1789760763.105344:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744270535598 time 744331843972 state 0x2 ref 0 00000100:00100000:0.0:1789760763.112422:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760763.112504:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298816 00000100:00100000:3.0:1789760763.112513:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298816 00000100:00100000:3.0:1789760763.112563:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744270535598 time 744331843972 state 0x0 ref 1 00000100:00100000:3.0:1789760763.112578:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744332104388 time 744331843972 state 0x1 ref 1 00000100:00100000:3.0:1789760763.112604:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744332104388 time 744427907470 state 0x0 ref 1 00000100:00100000:3.0:1789760763.112614:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507298816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760763.112672:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21295104, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:3.0:1789760763.112681:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.112729:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760763.112734:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.112741:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.112756:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.153840:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972519 is committed 00002000:00010000:1.0:1789760763.154769:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:1.0:1789760763.154787:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154800:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154819:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154832:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760763.154836:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154844:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154864:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.154872:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8a64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.154961:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507298816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42343us (42542us total) trans 4294972519 rc 0/0 00000100:00100000:1.0:1789760763.154970:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5229 00000100:00100000:1.0:1789760763.154980:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744332104388 time 744427907470 state 0x2 ref 0 00000100:00100000:2.0:1789760763.162238:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760763.162368:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507298944 00000100:00100000:3.0:1789760763.162379:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507298944 00000100:00100000:3.0:1789760763.162433:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744332104388 time 744427907470 state 0x0 ref 1 00000100:00100000:3.0:1789760763.162448:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744428167886 time 744427907470 state 0x1 ref 1 00000100:00100000:3.0:1789760763.162479:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744428167886 time 744477782120 state 0x0 ref 1 00000100:00100000:3.0:1789760763.162492:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507298944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760763.162543:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21299200, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:3.0:1789760763.162553:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.162568:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760763.162572:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.162580:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.162593:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.198226:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21299200 -> 21303296 00010000:00010000:0.0:1789760763.198248:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198261:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198271:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198282:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.198286:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198295:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198315:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.198322:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8a6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.198656:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507298944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36160us (36421us total) trans 4294972520 rc 0/0 00000100:00100000:0.0:1789760763.198676:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5230 00000100:00100000:0.0:1789760763.198686:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744428167886 time 744477782120 state 0x2 ref 0 00000001:00080000:0.0:1789760763.198766:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00000100:00100000:0.0:1789760763.208842:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.208932:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299072 00000100:00100000:1.0:1789760763.208941:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299072 00000100:00100000:1.0:1789760763.208989:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744428167886 time 744477782120 state 0x0 ref 1 00000100:00100000:1.0:1789760763.209004:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744478042536 time 744477782120 state 0x1 ref 1 00000100:00100000:1.0:1789760763.209034:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744478042536 time 744524337610 state 0x0 ref 1 00000100:00100000:1.0:1789760763.209048:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507299072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.209094:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21303296, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.209113:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.209129:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.209133:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.209139:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.209153:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.239574:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00002000:00010000:1.0:1789760763.239734:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21303296 -> 21307392 00010000:00010000:1.0:1789760763.239748:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239762:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239771:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239778:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760763.239783:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239793:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239812:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.239819:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8a72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.239902:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507299072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30851us (31065us total) trans 4294972521 rc 0/0 00000100:00100000:1.0:1789760763.239912:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5231 00000100:00100000:1.0:1789760763.239921:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744478042536 time 744524337610 state 0x2 ref 0 00000100:00100000:3.0:1789760763.245175:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.245481:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299200 00000100:00100000:1.0:1789760763.245491:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299200 00000100:00100000:1.0:1789760763.245548:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744478042536 time 744524337610 state 0x0 ref 1 00000100:00100000:1.0:1789760763.245562:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744524598026 time 744524337610 state 0x1 ref 1 00000100:00100000:1.0:1789760763.245645:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744524598026 time 744560948215 state 0x0 ref 1 00000100:00100000:1.0:1789760763.245658:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507299200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.245750:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21307392, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.245759:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245774:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.245779:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.245785:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245798:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.274522:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00002000:00010000:0.0:1789760763.274781:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:0.0:1789760763.274796:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274812:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274820:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274827:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.274833:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274842:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274862:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.274868:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8a79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.274966:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507299200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29305us (29795us total) trans 4294972522 rc 0/0 00000100:00100000:0.0:1789760763.274977:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5232 00000100:00100000:0.0:1789760763.274987:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744524598026 time 744560948215 state 0x2 ref 0 00000100:00100000:2.0:1789760763.279085:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.279213:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299328 00000100:00100000:0.0:1789760763.279223:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299328 00000100:00100000:0.0:1789760763.279276:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744524598026 time 744560948215 state 0x0 ref 1 00000100:00100000:0.0:1789760763.279290:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744561208631 time 744560948215 state 0x1 ref 1 00000100:00100000:0.0:1789760763.279321:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744561208631 time 744594624664 state 0x0 ref 1 00000100:00100000:0.0:1789760763.279333:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507299328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.279379:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21311488, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.279388:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.279404:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760763.279408:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.279425:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.279439:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.312791:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21311488 -> 21315584 00010000:00010000:0.0:1789760763.312812:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312826:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312842:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312849:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.312854:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312863:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312884:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.312891:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.312981:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507299328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33644us (33901us total) trans 4294972523 rc 0/0 00000100:00100000:0.0:1789760763.312991:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5233 00000100:00100000:0.0:1789760763.313001:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744561208631 time 744594624664 state 0x2 ref 0 00000001:00080000:2.0:1789760763.313525:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00000100:00100000:3.0:1789760763.327589:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.328508:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299456 00000100:00100000:1.0:1789760763.328521:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299456 00000100:00100000:1.0:1789760763.328581:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744561208631 time 744594624664 state 0x0 ref 1 00000100:00100000:1.0:1789760763.328594:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744594885080 time 744594624664 state 0x1 ref 1 00000100:00100000:0.0:1789760763.329499:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744594885080 time 744644795328 state 0x0 ref 1 00000100:00100000:0.0:1789760763.329519:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507299456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.329572:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21315584, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.329581:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.329599:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760763.329604:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.329611:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.329638:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.363212:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:0.0:1789760763.363234:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363248:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363257:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363265:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.363270:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363280:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363301:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.363309:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.363452:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972524 is committed 00000100:00100000:0.0:1789760763.363524:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507299456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34005us (35940us total) trans 4294972524 rc 0/0 00000100:00100000:0.0:1789760763.363535:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5234 00000100:00100000:0.0:1789760763.363547:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744594885080 time 744644795328 state 0x2 ref 0 00000100:00100000:3.0:1789760763.366910:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.367017:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299584 00000100:00100000:0.0:1789760763.367027:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299584 00000100:00100000:0.0:1789760763.367095:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744594885080 time 744644795328 state 0x0 ref 1 00000100:00100000:0.0:1789760763.367110:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744645055744 time 744644795328 state 0x1 ref 1 00000100:00100000:0.0:1789760763.367137:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744645055744 time 744682440526 state 0x0 ref 1 00000100:00100000:0.0:1789760763.367150:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507299584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.367196:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21319680, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.367205:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.367221:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760763.367226:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.367233:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.367247:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.399934:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972525 is committed 00002000:00010000:0.0:1789760763.400032:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21319680 -> 21323776 00010000:00010000:0.0:1789760763.400047:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400061:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400140:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400149:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.400154:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400162:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400182:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.400190:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8a8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.400322:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507299584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33169us (33418us total) trans 4294972525 rc 0/0 00000100:00100000:0.0:1789760763.400333:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5235 00000100:00100000:0.0:1789760763.400343:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744645055744 time 744682440526 state 0x2 ref 0 00000100:00100000:2.0:1789760763.404336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.404440:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299712 00000100:00100000:0.0:1789760763.404451:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299712 00000100:00100000:0.0:1789760763.404507:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744645055744 time 744682440526 state 0x0 ref 1 00000100:00100000:0.0:1789760763.404523:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744682700942 time 744682440526 state 0x1 ref 1 00000100:00100000:0.0:1789760763.404587:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744682700942 time 744719889949 state 0x0 ref 1 00000100:00100000:0.0:1789760763.404600:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507299712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.404860:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21323776, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.404871:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.404886:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760763.404891:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.404898:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.404914:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.444032:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21323776 -> 21327872 00010000:00010000:1.0:1789760763.444052:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444066:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444084:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444092:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760763.444098:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444107:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444126:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.444134:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8a95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.444228:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507299712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39624us (39900us total) trans 4294972526 rc 0/0 00000100:00100000:1.0:1789760763.444238:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5236 00000100:00100000:1.0:1789760763.444248:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744682700942 time 744719889949 state 0x2 ref 0 00000001:00080000:2.0:1789760763.448578:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972526 is committed 00000100:00100000:0.0:1789760763.448895:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760763.448988:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299840 00000100:00100000:2.0:1789760763.448997:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299840 00000100:00100000:2.0:1789760763.449049:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744682700942 time 744719889949 state 0x0 ref 1 00000100:00100000:2.0:1789760763.449065:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744720150365 time 744719889949 state 0x1 ref 1 00000100:00100000:2.0:1789760763.449210:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744720150365 time 744764511948 state 0x0 ref 1 00000100:00100000:2.0:1789760763.449224:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507299840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760763.449279:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21327872, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:2.0:1789760763.449481:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.449498:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760763.449504:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.449511:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.449525:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.488346:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972527 is committed 00002000:00010000:3.0:1789760763.488798:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:3.0:1789760763.488816:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488830:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488840:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488849:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760763.488854:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488863:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488883:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.488890:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8a9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.488978:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507299840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39751us (40082us total) trans 4294972527 rc 0/0 00000100:00100000:3.0:1789760763.488988:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5237 00000100:00100000:3.0:1789760763.488998:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744720150365 time 744764511948 state 0x2 ref 0 00000100:00100000:0.0:1789760763.494188:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.494271:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507299968 00000100:00100000:1.0:1789760763.494280:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507299968 00000100:00100000:1.0:1789760763.494328:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744720150365 time 744764511948 state 0x0 ref 1 00000100:00100000:1.0:1789760763.494343:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744764772364 time 744764511948 state 0x1 ref 1 00000100:00100000:1.0:1789760763.494423:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744764772364 time 744809725194 state 0x0 ref 1 00000100:00100000:1.0:1789760763.494436:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507299968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.494485:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21331968, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.494495:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.494510:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.494514:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.494521:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.494535:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.536591:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:0.0:1789760763.536609:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536631:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536641:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536648:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.536653:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536668:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536687:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.536718:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aa3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.536797:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507299968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42359us (42616us total) trans 4294972528 rc 0/0 00000100:00100000:0.0:1789760763.536807:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5238 00000100:00100000:0.0:1789760763.536816:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744764772364 time 744809725194 state 0x2 ref 0 00000001:00080000:0.0:1789760763.536850:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972528 is committed 00000100:00100000:0.0:1789760763.543753:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.543834:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300096 00000100:00100000:1.0:1789760763.543845:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300096 00000100:00100000:1.0:1789760763.543897:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744764772364 time 744809725194 state 0x0 ref 1 00000100:00100000:1.0:1789760763.543912:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744809985610 time 744809725194 state 0x1 ref 1 00000100:00100000:1.0:1789760763.543972:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744809985610 time 744859275719 state 0x0 ref 1 00000100:00100000:1.0:1789760763.543982:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507300096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.544028:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21336064, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.544037:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.544053:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.544058:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.544064:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.544087:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.582830:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:0.0:1789760763.583109:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21336064 -> 21340160 00010000:00010000:0.0:1789760763.583124:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583136:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583146:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583154:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.583159:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583169:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583189:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.583198:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8aaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.583292:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507300096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39305us (39542us total) trans 4294972529 rc 0/0 00000100:00100000:0.0:1789760763.583302:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5239 00000100:00100000:0.0:1789760763.583313:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744809985610 time 744859275719 state 0x2 ref 0 00000100:00100000:2.0:1789760763.591345:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760763.591434:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300224 00000100:00100000:3.0:1789760763.591445:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300224 00000100:00100000:3.0:1789760763.591502:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744809985610 time 744859275719 state 0x0 ref 1 00000100:00100000:3.0:1789760763.591517:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744859536135 time 744859275719 state 0x1 ref 1 00000100:00100000:3.0:1789760763.591544:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744859536135 time 744906847602 state 0x0 ref 1 00000100:00100000:3.0:1789760763.591555:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507300224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760763.591613:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21340160, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:3.0:1789760763.591630:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.591645:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760763.591649:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.591655:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.591675:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.612965:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.613013:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.613036:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.613054:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760763.613396:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300480 00000100:00100000:3.0:1789760763.613406:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300480 00000100:00100000:3.0:1789760763.613472:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300608 00000100:00100000:3.0:1789760763.613476:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300608 00000100:00100000:3.0:1789760763.613495:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300736 00000100:00100000:3.0:1789760763.613498:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300736 00000100:00100000:3.0:1789760763.613531:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 928 00000100:00100000:3.0:1789760763.613541:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507300480:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760763.613602:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 929 00000100:00100000:1.0:1789760763.613610:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507300608:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760763.613811:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760763.613931:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300864 00000100:00100000:2.0:1789760763.613935:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300864 00000100:00100000:2.0:1789760763.613960:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 930 00000100:00100000:2.0:1789760763.613967:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507300736:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760763.614194:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507300480:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 653us (1237us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.614202:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 928 00000100:00100000:3.0:1789760763.614218:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 931 00000100:00100000:3.0:1789760763.614225:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507300864:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760763.614251:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105820a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507300864:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 27us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.614256:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 931 00000100:00080000:3.0:1789760763.614993:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760763.614999:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760763.615049:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300352 00000100:00100000:0.0:1789760763.615054:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300352 00000100:00100000:0.0:1789760763.615175:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 476 00000100:00100000:0.0:1789760763.615183:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507300352:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760763.615227:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507300352:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 46us (2213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760763.615235:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 476 00000100:00100000:1.0:1789760763.619373:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507300608:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 5753us (6324us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760763.619387:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 929 00000100:00100000:2.0:1789760763.619743:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507300736:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 5774us (6684us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760763.619751:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 930 00000001:00080000:0.0:1789760763.630231:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00002000:00010000:2.0:1789760763.630238:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21340160 -> 21344256 00010000:00010000:2.0:1789760763.630252:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630265:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630274:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630282:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760763.630285:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630292:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630390:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.630396:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8ab1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.630486:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507300224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38928us (39146us total) trans 4294972530 rc 0/0 00000100:00100000:2.0:1789760763.630493:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5240 00000100:00100000:2.0:1789760763.630762:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744859536135 time 744906847602 state 0x2 ref 0 00000100:00100000:0.0:1789760763.633379:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.633506:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507300992 00000100:00100000:1.0:1789760763.633515:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507300992 00000100:00100000:1.0:1789760763.633566:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744859536135 time 744906847602 state 0x0 ref 1 00000100:00100000:1.0:1789760763.633582:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744907108018 time 744906847602 state 0x1 ref 1 00000100:00100000:1.0:1789760763.633613:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744907108018 time 744948915882 state 0x0 ref 1 00000100:00100000:1.0:1789760763.633636:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507300992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.633683:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21344256, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.633720:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.633736:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.633740:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.633747:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.633762:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.674509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:0.0:1789760763.675057:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:0.0:1789760763.675073:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675091:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675100:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675108:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.675113:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675123:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675144:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.675151:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8ab8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.675444:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507300992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41815us (42067us total) trans 4294972531 rc 0/0 00000100:00100000:0.0:1789760763.675455:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5241 00000100:00100000:0.0:1789760763.675465:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744907108018 time 744948915882 state 0x2 ref 0 00000100:00100000:0.0:1789760763.678099:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760763.678165:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301120 00000100:00100000:1.0:1789760763.678176:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301120 00000100:00100000:1.0:1789760763.678228:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744907108018 time 744948915882 state 0x0 ref 1 00000100:00100000:1.0:1789760763.678242:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744949176298 time 744948915882 state 0x1 ref 1 00000100:00100000:1.0:1789760763.678275:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744949176298 time 744993578173 state 0x0 ref 1 00000100:00100000:1.0:1789760763.678287:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507301120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.678334:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21348352, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.678343:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.678358:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760763.678363:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.678370:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.678383:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.698590:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00002000:00010000:0.0:1789760763.698722:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:0.0:1789760763.698741:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698755:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698765:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698773:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.698778:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698787:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698810:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.698817:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8abf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.698899:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507301120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 20609us (20804us total) trans 4294972532 rc 0/0 00000100:00100000:0.0:1789760763.698908:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5242 00000100:00100000:0.0:1789760763.698918:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744949176298 time 744993578173 state 0x2 ref 0 00000100:00100000:3.0:1789760763.703296:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760763.703554:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301248 00000100:00100000:2.0:1789760763.703564:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301248 00000100:00100000:2.0:1789760763.703630:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744949176298 time 744993578173 state 0x0 ref 1 00000100:00100000:2.0:1789760763.703646:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744993838589 time 744993578173 state 0x1 ref 1 00000100:00100000:2.0:1789760763.703898:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744993838589 time 745019200161 state 0x0 ref 1 00000100:00100000:2.0:1789760763.703917:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507301248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760763.703965:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21352448, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:2.0:1789760763.703974:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.703991:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760763.703995:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.704001:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.704016:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.732326:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:2.0:1789760763.732348:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732363:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732373:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732382:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760763.732387:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732397:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732418:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.732426:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8ac6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.732522:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507301248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28603us (29231us total) trans 4294972533 rc 0/0 00000100:00100000:2.0:1789760763.732533:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5243 00000100:00100000:2.0:1789760763.732543:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744993838589 time 745019200161 state 0x2 ref 0 00000001:00080000:0.0:1789760763.734205:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00000100:00100000:1.0:1789760763.739008:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.739140:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301376 00000100:00100000:0.0:1789760763.739149:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301376 00000100:00100000:0.0:1789760763.739204:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 744993838589 time 745019200161 state 0x0 ref 1 00000100:00100000:0.0:1789760763.739235:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745019460577 time 745019200161 state 0x1 ref 1 00000100:00100000:0.0:1789760763.739309:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745019460577 time 745054611864 state 0x0 ref 1 00000100:00100000:0.0:1789760763.739323:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507301376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.739371:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21356544, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.739381:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.739397:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760763.739403:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.739409:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.739424:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.790752:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:1.0:1789760763.790777:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790791:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790800:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790807:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760763.790811:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790819:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790837:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.790844:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8acd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.791155:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507301376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51825us (52147us total) trans 4294972534 rc 0/0 00000100:00100000:1.0:1789760763.791166:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5244 00000100:00100000:1.0:1789760763.791176:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745019460577 time 745054611864 state 0x2 ref 0 00000001:00080000:3.0:1789760763.791260:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00000100:00100000:1.0:1789760763.799948:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.800201:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301504 00000100:00100000:0.0:1789760763.800210:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301504 00000100:00100000:0.0:1789760763.800267:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745019460577 time 745054611864 state 0x0 ref 1 00000100:00100000:0.0:1789760763.800282:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745054872280 time 745054611864 state 0x1 ref 1 00000100:00100000:0.0:1789760763.800313:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745054872280 time 745115615664 state 0x0 ref 1 00000100:00100000:0.0:1789760763.800325:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507301504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.800373:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21360640, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.800382:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.800397:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760763.800402:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.800408:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.800423:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.804595:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760763.863250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972535 is committed 00002000:00010000:0.0:1789760763.863361:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21360640 -> 21364736 00010000:00010000:0.0:1789760763.863378:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863392:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863401:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863412:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.863417:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863425:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863447:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.863456:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ad4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.863579:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507301504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63251us (63635us total) trans 4294972535 rc 0/0 00000100:00100000:0.0:1789760763.863590:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5245 00000100:00100000:0.0:1789760763.863600:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745054872280 time 745115615664 state 0x2 ref 0 00000100:00100000:0.0:1789760763.869393:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760763.869914:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301632 00000100:00100000:2.0:1789760763.869925:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301632 00000100:00100000:2.0:1789760763.869988:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745054872280 time 745115615664 state 0x0 ref 1 00000100:00100000:2.0:1789760763.870003:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745115876080 time 745115615664 state 0x1 ref 1 00000100:00100000:2.0:1789760763.870081:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745115876080 time 745185383508 state 0x0 ref 1 00000100:00100000:2.0:1789760763.870095:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507301632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760763.870144:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21364736, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:2.0:1789760763.870153:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.870170:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760763.870175:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.870181:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.870195:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.897753:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972536 is committed 00002000:00010000:3.0:1789760763.897849:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:3.0:1789760763.897864:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897876:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897885:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897892:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760763.897897:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897906:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897926:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.897934:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8adb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.898298:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507301632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28199us (28911us total) trans 4294972536 rc 0/0 00000100:00100000:3.0:1789760763.898310:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5246 00000100:00100000:3.0:1789760763.898320:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745115876080 time 745185383508 state 0x2 ref 0 00000100:00100000:1.0:1789760763.902239:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.902351:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301760 00000100:00100000:0.0:1789760763.902361:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301760 00000100:00100000:0.0:1789760763.902417:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745115876080 time 745185383508 state 0x0 ref 1 00000100:00100000:0.0:1789760763.902432:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745185643924 time 745185383508 state 0x1 ref 1 00000100:00100000:0.0:1789760763.902496:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745185643924 time 745217798984 state 0x0 ref 1 00000100:00100000:0.0:1789760763.902508:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507301760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760763.902561:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21368832, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:0.0:1789760763.902571:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.902586:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760763.902591:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.902598:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.902611:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.937543:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00002000:00010000:0.0:1789760763.937637:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:0.0:1789760763.937654:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937755:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937765:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937772:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760763.937777:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937786:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937809:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.937817:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b8ae2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.937962:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507301760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35441us (35719us total) trans 4294972537 rc 0/0 00000100:00100000:0.0:1789760763.937974:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5247 00000100:00100000:0.0:1789760763.937984:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745185643924 time 745217798984 state 0x2 ref 0 00000100:00100000:1.0:1789760763.942609:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760763.943195:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507301888 00000100:00100000:0.0:1789760763.943207:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507301888 00000100:00100000:1.0:1789760763.943447:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745185643924 time 745217798984 state 0x0 ref 1 00000100:00100000:1.0:1789760763.943462:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745218059400 time 745217798984 state 0x1 ref 1 00000100:00100000:1.0:1789760763.943521:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745218059400 time 745258823994 state 0x0 ref 1 00000100:00100000:1.0:1789760763.943534:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507301888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760763.943584:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21372928, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760763.943592:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.943608:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760763.943612:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.943629:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.943648:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.974608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00002000:00010000:3.0:1789760763.974741:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21372928 -> 21377024 00010000:00010000:3.0:1789760763.974756:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974768:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974778:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974791:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760763.974796:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974805:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974824:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.974831:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b8ae9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.974917:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507301888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31381us (32312us total) trans 4294972538 rc 0/0 00000100:00100000:3.0:1789760763.974928:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5248 00000100:00100000:3.0:1789760763.974938:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745218059400 time 745258823994 state 0x2 ref 0 00000100:00100000:3.0:1789760763.979382:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760763.979525:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302016 00000100:00100000:2.0:1789760763.979535:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302016 00000100:00100000:2.0:1789760763.979590:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745218059400 time 745258823994 state 0x0 ref 1 00000100:00100000:2.0:1789760763.979606:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745259084410 time 745258823994 state 0x1 ref 1 00000100:00100000:2.0:1789760763.979644:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745259084410 time 745294946886 state 0x0 ref 1 00000100:00100000:2.0:1789760763.979662:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507302016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760763.979742:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21377024, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:2.0:1789760763.979751:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.979767:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760763.979772:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.979779:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.979798:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.023530:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00002000:00010000:3.0:1789760764.024122:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21377024 -> 21381120 00010000:00010000:3.0:1789760764.024142:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024157:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024172:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024179:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760764.024183:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024202:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024221:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.024228:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8af0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.024332:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507302016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44672us (44955us total) trans 4294972539 rc 0/0 00000100:00100000:3.0:1789760764.024342:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5249 00000100:00100000:3.0:1789760764.024352:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745259084410 time 745294946886 state 0x2 ref 0 00000100:00100000:0.0:1789760764.034602:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760764.034733:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302144 00000100:00100000:1.0:1789760764.034743:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302144 00000100:00100000:1.0:1789760764.034798:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745259084410 time 745294946886 state 0x0 ref 1 00000100:00100000:1.0:1789760764.034814:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745295207302 time 745294946886 state 0x1 ref 1 00000100:00100000:1.0:1789760764.034834:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745295207302 time 745350138356 state 0x0 ref 1 00000100:00100000:1.0:1789760764.034847:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.034904:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21381120, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xa807 00010000:00010000:1.0:1789760764.034913:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.034927:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760764.034932:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.034937:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.034951:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.096904:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00002000:00010000:3.0:1789760764.097101:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21381120 -> 21385216 00002000:00010000:3.0:1789760764.097106:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:3.0:1789760764.097109:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:3.0:1789760764.097125:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097138:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097147:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097154:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760764.097159:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097167:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097185:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.097194:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8af7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.097375:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62526us (62777us total) trans 4294972540 rc 0/0 00000100:00100000:3.0:1789760764.097386:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5250 00000100:00100000:3.0:1789760764.097395:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745295207302 time 745350138356 state 0x2 ref 0 00000100:00100000:0.0:1789760764.100848:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.101017:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302272 00000100:00100000:3.0:1789760764.101027:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302272 00000100:00100000:3.0:1789760764.101079:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745295207302 time 745350138356 state 0x0 ref 1 00000100:00100000:3.0:1789760764.101094:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745350398772 time 745350138356 state 0x1 ref 1 00000100:00100000:3.0:1789760764.101153:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745350398772 time 745416455135 state 0x0 ref 1 00000100:00100000:3.0:1789760764.101166:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.101213:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21385216, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.101223:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.101237:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.101241:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.101247:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.101261:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.123976:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538283008:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760764.124043:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124099:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538283520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124123:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124144:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234677b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538284032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124168:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124186:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1051c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538284544:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124205:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124227:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789760764.124232:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538283136:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124253:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124271:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234677100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538283648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124291:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124308:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234676d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538284160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124327:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124341:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1051cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538284672:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124361:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124390:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538283264:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124425:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124443:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538283776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124465:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124482:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234675180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538284288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124502:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124520:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760764.124525:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538283392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124544:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124561:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234675500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538283904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124581:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124599:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1051c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538284416:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.124628:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.124783:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283008 00000100:00100000:3.0:1789760764.124790:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283008 00000100:00100000:3.0:1789760764.124839:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789760764.124849:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538283008:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760764.124882:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760764.124926:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538283008:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (882us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.124933:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789760764.124956:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283520 00000100:00100000:3.0:1789760764.124959:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283520 00000100:00100000:3.0:1789760764.124977:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283264 00000100:00100000:3.0:1789760764.124980:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283264 00000100:00100000:3.0:1789760764.124990:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283776 00000100:00100000:3.0:1789760764.124992:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283776 00000100:00100000:3.0:1789760764.125006:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284288 00000100:00100000:3.0:1789760764.125008:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284288 00000100:00100000:3.0:1789760764.125020:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283904 00000100:00100000:3.0:1789760764.125022:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283904 00000100:00100000:3.0:1789760764.125034:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284416 00000100:00100000:3.0:1789760764.125036:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284416 00000100:00100000:3.0:1789760764.125050:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 932 00000100:00100000:3.0:1789760764.125056:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538283520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125089:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538283520:12345-0@lo:400:kworker.0 Request processed in 33us (965us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125096:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 932 00000100:00100000:3.0:1789760764.125109:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 933 00000100:00100000:3.0:1789760764.125115:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538283264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125146:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538283264:12345-0@lo:400:kworker.0 Request processed in 31us (720us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125155:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 933 00000100:00100000:3.0:1789760764.125165:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 934 00000100:00100000:3.0:1789760764.125171:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538283776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125199:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538283776:12345-0@lo:400:kworker.0 Request processed in 28us (732us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125203:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 934 00000100:00100000:3.0:1789760764.125213:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760764.125219:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538284288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125239:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538284288:12345-0@lo:400:kworker.0 Request processed in 22us (737us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125244:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760764.125255:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760764.125260:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538283904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125284:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538283904:12345-0@lo:400:kworker.0 Request processed in 25us (702us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125290:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760764.125301:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1789760764.125307:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538284416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.125331:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538284416:12345-0@lo:400:kworker.0 Request processed in 25us (712us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.125337:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1789760764.125354:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284032 00000100:00100000:3.0:1789760764.125358:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284032 00000100:00100000:3.0:1789760764.125381:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284544 00000100:00100000:3.0:1789760764.125384:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284544 00000100:00100000:3.0:1789760764.125394:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284160 00000100:00100000:3.0:1789760764.125397:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284160 00000100:00100000:3.0:1789760764.126523:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284672 00000100:00100000:3.0:1789760764.126531:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284672 00000100:00100000:3.0:1789760764.126559:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789760764.126567:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538284032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.126636:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538284032:12345-0@lo:400:kworker.0 Request processed in 71us (2467us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.126642:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789760764.126660:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:3.0:1789760764.126667:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538284544:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.126725:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538284544:12345-0@lo:400:kworker.0 Request processed in 58us (2518us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.126731:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000100:00100000:3.0:1789760764.126742:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:3.0:1789760764.126748:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538284160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.126776:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538284160:12345-0@lo:400:kworker.0 Request processed in 29us (2448us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.126782:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:3.0:1789760764.126792:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1789760764.126798:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538284672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.126824:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538284672:12345-0@lo:400:kworker.0 Request processed in 26us (2461us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.126829:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1789760764.126852:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283136 00000100:00100000:3.0:1789760764.126856:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283136 00000100:00100000:3.0:1789760764.126879:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789760764.126886:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538283136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.126916:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538283136:12345-0@lo:400:kworker.0 Request processed in 29us (2661us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.126922:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789760764.126950:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283648 00000100:00100000:3.0:1789760764.126954:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283648 00000100:00100000:3.0:1789760764.126979:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538283392 00000100:00100000:3.0:1789760764.126982:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538283392 00000100:00100000:3.0:1789760764.126996:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789760764.127001:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538283648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127033:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538283648:12345-0@lo:400:kworker.0 Request processed in 31us (2740us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.127039:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789760764.127049:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:3.0:1789760764.127055:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f41ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538283392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127079:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f41ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538283392:12345-0@lo:400:kworker.0 Request processed in 25us (2534us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.127084:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00080000:3.0:1789760764.127125:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760764.127130:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760764.127132:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760764.127135:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760764.127151:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760764.127164:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538284800:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760764.127196:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.127227:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284800 00000100:00100000:3.0:1789760764.127231:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284800 00000100:00100000:3.0:1789760764.127252:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:3.0:1789760764.127258:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538284800:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760764.127272:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760764.127301:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538284800:12345-0@lo:13:osp-pre-1-0.0 Request processed in 42us (104us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.127306:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:3.0:1789760764.127340:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538284800:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760764.127367:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538283392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127377:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234675500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538283904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127387:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1051c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538284416:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127396:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538283264:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127405:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234676a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538283776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127414:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234675180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538284288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127443:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538283136:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127453:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234677100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538283648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127462:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234676d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538284160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127470:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1051cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538284672:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127483:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538283008:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760764.127491:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538283520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127499:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234677b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538284032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.127507:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1051c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538284544:0@lo:400:kworker.0 00000001:00080000:2.0:1789760764.134735:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00002000:00010000:0.0:1789760764.134830:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21385216 -> 21389312 00010000:00010000:0.0:1789760764.134845:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134864:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134872:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134880:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.134885:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134893:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134912:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.134920:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8afe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.135009:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33840us (34171us total) trans 4294972541 rc 0/0 00000100:00100000:0.0:1789760764.135019:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5251 00000100:00100000:0.0:1789760764.135028:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745350398772 time 745416455135 state 0x2 ref 0 00000100:00100000:0.0:1789760764.138515:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.138975:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302400 00000100:00100000:3.0:1789760764.138986:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302400 00000100:00100000:3.0:1789760764.139041:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745350398772 time 745416455135 state 0x0 ref 1 00000100:00100000:3.0:1789760764.139056:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745416715551 time 745416455135 state 0x1 ref 1 00000100:00100000:3.0:1789760764.139117:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745416715551 time 745454420103 state 0x0 ref 1 00000100:00100000:3.0:1789760764.139129:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507302400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.139175:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21389312, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.139184:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.139198:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.139202:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.139208:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.139222:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.171636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00002000:00010000:0.0:1789760764.171650:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21389312 -> 21393408 00010000:00010000:0.0:1789760764.171674:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171686:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171726:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171733:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.171738:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171747:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171767:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.171775:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8b05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.171916:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507302400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32785us (33408us total) trans 4294972542 rc 0/0 00000100:00100000:1.0:1789760764.171925:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5252 00000100:00100000:1.0:1789760764.171938:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745416715551 time 745454420103 state 0x2 ref 0 00000100:00100000:1.0:1789760764.178800:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.178870:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302528 00000100:00100000:3.0:1789760764.178880:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302528 00000100:00100000:3.0:1789760764.178933:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745416715551 time 745454420103 state 0x0 ref 1 00000100:00100000:3.0:1789760764.178948:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745454680519 time 745454420103 state 0x1 ref 1 00000100:00100000:3.0:1789760764.178979:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745454680519 time 745494282272 state 0x0 ref 1 00000100:00100000:3.0:1789760764.178991:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.179108:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21393408, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.179118:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.179133:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.179137:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.179143:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.179156:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.212898:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:0.0:1789760764.212916:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212934:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212943:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212951:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.212955:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212964:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212981:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.212988:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8b0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.213077:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507302528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34083us (34283us total) trans 4294972543 rc 0/0 00000100:00100000:0.0:1789760764.213087:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5253 00000100:00100000:0.0:1789760764.213097:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745454680519 time 745494282272 state 0x2 ref 0 00000001:00080000:2.0:1789760764.213566:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00000100:00100000:0.0:1789760764.219570:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.219759:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302656 00000100:00100000:3.0:1789760764.219771:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302656 00000100:00100000:3.0:1789760764.219836:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745454680519 time 745494282272 state 0x0 ref 1 00000100:00100000:3.0:1789760764.219855:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745494542688 time 745494282272 state 0x1 ref 1 00000100:00100000:3.0:1789760764.219932:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745494542688 time 745535191837 state 0x0 ref 1 00000100:00100000:3.0:1789760764.219952:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507302656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.220033:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21397504, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.220044:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220190:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.220195:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.220204:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220223:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.265952:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21397504 -> 21401600 00010000:00010000:0.0:1789760764.265981:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.265998:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.266007:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.266014:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.266019:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.266029:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.266052:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.266060:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8b13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.266192:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507302656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46237us (46681us total) trans 4294972544 rc 0/0 00000100:00100000:0.0:1789760764.266203:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5254 00000100:00100000:0.0:1789760764.266216:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745494542688 time 745535191837 state 0x2 ref 0 00000001:00080000:1.0:1789760764.267369:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00000100:00100000:3.0:1789760764.269567:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760764.269760:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302784 00000100:00100000:1.0:1789760764.269773:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302784 00000100:00100000:1.0:1789760764.269847:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745494542688 time 745535191837 state 0x0 ref 1 00000100:00100000:1.0:1789760764.269866:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745535452253 time 745535191837 state 0x1 ref 1 00000100:00100000:1.0:1789760764.269998:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745535452253 time 745585300039 state 0x0 ref 1 00000100:00100000:1.0:1789760764.270015:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.270090:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21401600, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:1.0:1789760764.270101:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.270121:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760764.270126:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.270134:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.270151:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.309097:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00002000:00010000:1.0:1789760764.309261:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21401600 -> 21405696 00010000:00010000:1.0:1789760764.309280:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309295:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309306:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309314:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.309319:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309329:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309354:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.309360:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.309474:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39456us (39915us total) trans 4294972545 rc 0/0 00000100:00100000:1.0:1789760764.309490:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5255 00000100:00100000:1.0:1789760764.309503:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745535452253 time 745585300039 state 0x2 ref 0 00000100:00100000:3.0:1789760764.312300:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760764.312462:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507302912 00000100:00100000:0.0:1789760764.312475:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507302912 00000100:00100000:0.0:1789760764.312547:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745535452253 time 745585300039 state 0x0 ref 1 00000100:00100000:0.0:1789760764.312567:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745585560455 time 745585300039 state 0x1 ref 1 00000100:00100000:0.0:1789760764.312609:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745585560455 time 745627911727 state 0x0 ref 1 00000100:00100000:0.0:1789760764.312634:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760764.312737:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21405696, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:0.0:1789760764.312748:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.312768:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760764.312774:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.312782:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.312800:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.346050:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00002000:00010000:0.0:1789760764.346252:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21405696 -> 21409792 00010000:00010000:0.0:1789760764.346273:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346295:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346307:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346315:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.346320:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346331:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346355:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.346363:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.346529:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507302912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33891us (34234us total) trans 4294972546 rc 0/0 00000100:00100000:0.0:1789760764.346542:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5256 00000100:00100000:0.0:1789760764.346554:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745585560455 time 745627911727 state 0x2 ref 0 00000100:00100000:0.0:1789760764.350035:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.350514:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303040 00000100:00100000:3.0:1789760764.350528:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303040 00000100:00100000:3.0:1789760764.350605:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745585560455 time 745627911727 state 0x0 ref 1 00000100:00100000:3.0:1789760764.351218:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745628172143 time 745627911727 state 0x1 ref 1 00000100:00100000:3.0:1789760764.351256:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745628172143 time 745666559061 state 0x0 ref 1 00000100:00100000:3.0:1789760764.351276:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507303040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.351344:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21409792, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.351355:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.351379:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.351384:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.351391:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.351409:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.385379:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21409792 -> 21413888 00010000:00010000:0.0:1789760764.385407:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385423:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385435:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385443:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.385448:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385459:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385484:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.385493:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8b28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.385836:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00000100:00100000:0.0:1789760764.386157:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507303040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34877us (36126us total) trans 4294972547 rc 0/0 00000100:00100000:0.0:1789760764.386171:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5257 00000100:00100000:0.0:1789760764.386186:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745628172143 time 745666559061 state 0x2 ref 0 00000100:00100000:3.0:1789760764.393884:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760764.394007:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303168 00000100:00100000:1.0:1789760764.394021:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303168 00000100:00100000:1.0:1789760764.394103:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745628172143 time 745666559061 state 0x0 ref 1 00000100:00100000:1.0:1789760764.394122:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745666819477 time 745666559061 state 0x1 ref 1 00000100:00100000:1.0:1789760764.394198:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745666819477 time 745709500093 state 0x0 ref 1 00000100:00100000:1.0:1789760764.394214:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.394279:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21413888, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:1.0:1789760764.394291:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.394312:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760764.394317:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.394325:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.394343:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.449452:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00002000:00010000:1.0:1789760764.449628:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21413888 -> 21417984 00010000:00010000:1.0:1789760764.449650:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449667:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449678:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449687:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.449731:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449743:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449773:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.449783:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.449910:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55693us (56033us total) trans 4294972548 rc 0/0 00000100:00100000:1.0:1789760764.449924:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5258 00000100:00100000:1.0:1789760764.449936:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745666819477 time 745709500093 state 0x2 ref 0 00000100:00100000:0.0:1789760764.457483:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760764.457599:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303296 00000100:00100000:3.0:1789760764.457613:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303296 00000100:00100000:3.0:1789760764.457817:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745666819477 time 745709500093 state 0x0 ref 1 00000100:00100000:3.0:1789760764.457838:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745709760509 time 745709500093 state 0x1 ref 1 00000100:00100000:3.0:1789760764.457878:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745709760509 time 745773180707 state 0x0 ref 1 00000100:00100000:3.0:1789760764.457901:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760764.457963:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21417984, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:3.0:1789760764.457974:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.457993:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760764.457998:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.458006:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.458024:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.514400:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00002000:00010000:2.0:1789760764.514402:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:2.0:1789760764.514426:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514441:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514451:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514459:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760764.514464:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514474:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514498:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.514510:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8b36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.514923:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57017us (57446us total) trans 4294972549 rc 0/0 00000100:00100000:2.0:1789760764.514936:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5259 00000100:00100000:2.0:1789760764.514948:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745709760509 time 745773180707 state 0x2 ref 0 00000100:00100000:2.0:1789760764.523326:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760764.523669:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303424 00000100:00100000:0.0:1789760764.523683:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303424 00000100:00100000:0.0:1789760764.523878:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745709760509 time 745773180707 state 0x0 ref 1 00000100:00100000:0.0:1789760764.524078:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745773441123 time 745773180707 state 0x1 ref 1 00000100:00100000:1.0:1789760764.525060:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745773441123 time 745840354373 state 0x0 ref 1 00000100:00100000:1.0:1789760764.525090:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.525303:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21422080, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:1.0:1789760764.525315:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.525337:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760764.525343:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.525351:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.525369:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.586612:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00002000:00010000:1.0:1789760764.587100:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:1.0:1789760764.587122:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587134:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587143:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587150:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.587154:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587163:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587182:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.587189:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.587354:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62272us (64036us total) trans 4294972550 rc 0/0 00000100:00100000:1.0:1789760764.587366:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5260 00000100:00100000:1.0:1789760764.587375:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745773441123 time 745840354373 state 0x2 ref 0 00000100:00100000:3.0:1789760764.599563:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760764.599910:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303552 00000100:00100000:1.0:1789760764.599936:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303552 00000100:00100000:1.0:1789760764.599999:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745773441123 time 745840354373 state 0x0 ref 1 00000100:00100000:1.0:1789760764.600016:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745840614789 time 745840354373 state 0x1 ref 1 00000100:00100000:1.0:1789760764.600052:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745840614789 time 745915355664 state 0x0 ref 1 00000100:00100000:1.0:1789760764.600066:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.600117:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21426176, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:1.0:1789760764.600126:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.600143:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760764.600147:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.600153:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.600168:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.663867:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00002000:00010000:1.0:1789760764.663940:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:1.0:1789760764.663956:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.663968:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.663978:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.663985:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.663989:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.663998:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.664023:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.664031:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8b44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.664115:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64047us (64563us total) trans 4294972551 rc 0/0 00000100:00100000:1.0:1789760764.664125:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5261 00000100:00100000:1.0:1789760764.664135:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745840614789 time 745915355664 state 0x2 ref 0 00000100:00100000:0.0:1789760764.667880:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760764.668170:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303680 00000100:00100000:2.0:1789760764.668180:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303680 00000100:00100000:2.0:1789760764.668238:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745840614789 time 745915355664 state 0x0 ref 1 00000100:00100000:2.0:1789760764.668254:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745915616080 time 745915355664 state 0x1 ref 1 00000100:00100000:2.0:1789760764.668342:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745915616080 time 745983644765 state 0x0 ref 1 00000100:00100000:2.0:1789760764.668357:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760764.668405:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21430272, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:2.0:1789760764.668415:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.668431:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760764.668436:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.668442:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.668456:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.772886:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00002000:00010000:0.0:1789760764.772972:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:0.0:1789760764.772987:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773000:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773009:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773016:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.773022:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773032:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773055:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.773067:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8b4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.773284:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 104923us (105407us total) trans 4294972552 rc 0/0 00000100:00100000:1.0:1789760764.773296:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5262 00000100:00100000:1.0:1789760764.773306:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745915616080 time 745983644765 state 0x2 ref 0 00000100:00100000:1.0:1789760764.779214:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760764.782645:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303808 00000100:00100000:0.0:1789760764.782921:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303808 00000100:00100000:0.0:1789760764.782989:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745915616080 time 745983644765 state 0x0 ref 1 00000100:00100000:0.0:1789760764.783004:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745983905181 time 745983644765 state 0x1 ref 1 00000100:00100000:0.0:1789760764.783042:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745983905181 time 746098344753 state 0x0 ref 1 00000100:00100000:0.0:1789760764.783056:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760764.783106:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21434368, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:0.0:1789760764.783115:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.783133:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760764.783138:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.783145:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.783159:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.830491:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760764.859918:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:0.0:1789760764.860107:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:0.0:1789760764.860122:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860134:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860142:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860149:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760764.860153:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860162:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860180:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.860188:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.860281:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507303808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77223us (81073us total) trans 4294972553 rc 0/0 00000100:00100000:0.0:1789760764.860291:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5263 00000100:00100000:0.0:1789760764.860301:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745983905181 time 746098344753 state 0x2 ref 0 00000100:00100000:3.0:1789760764.871762:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760764.871882:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507303936 00000100:00100000:1.0:1789760764.871894:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507303936 00000100:00100000:1.0:1789760764.871949:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 745983905181 time 746098344753 state 0x0 ref 1 00000100:00100000:1.0:1789760764.871964:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746098605169 time 746098344753 state 0x1 ref 1 00000100:00100000:1.0:1789760764.871995:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746098605169 time 746187298025 state 0x0 ref 1 00000100:00100000:1.0:1789760764.872012:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760764.872066:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21438464, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:1.0:1789760764.872076:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.872093:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760764.872098:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.872105:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.872122:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.946793:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00002000:00010000:1.0:1789760764.947048:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:1.0:1789760764.947072:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947083:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947092:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947099:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.947103:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947113:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947132:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.947140:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.947229:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507303936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75215us (75471us total) trans 4294972554 rc 0/0 00000100:00100000:1.0:1789760764.947239:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5264 00000100:00100000:1.0:1789760764.947249:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746098605169 time 746187298025 state 0x2 ref 0 00000100:00100000:3.0:1789760764.955789:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760764.955950:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304064 00000100:00100000:0.0:1789760764.955959:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304064 00000100:00100000:0.0:1789760764.956014:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746098605169 time 746187298025 state 0x0 ref 1 00000100:00100000:0.0:1789760764.956030:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746187558441 time 746187298025 state 0x1 ref 1 00000100:00100000:0.0:1789760764.956065:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746187558441 time 746271367181 state 0x0 ref 1 00000100:00100000:0.0:1789760764.956079:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760764.956129:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21442560, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:0.0:1789760764.956138:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.956153:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760764.956158:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.956164:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.956179:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.996129:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00002000:00010000:1.0:1789760764.996353:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21442560 -> 21446656 00010000:00010000:1.0:1789760764.996368:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996382:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996391:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996409:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760764.996415:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996425:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996444:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.996452:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.996576:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40495us (40793us total) trans 4294972555 rc 0/0 00000100:00100000:1.0:1789760764.996589:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5265 00000100:00100000:1.0:1789760764.996598:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746187558441 time 746271367181 state 0x2 ref 0 00000100:00100000:2.0:1789760765.003129:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760765.003275:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304192 00000100:00100000:0.0:1789760765.003286:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304192 00000100:00100000:0.0:1789760765.003359:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746187558441 time 746271367181 state 0x0 ref 1 00000100:00100000:0.0:1789760765.003374:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746271627597 time 746271367181 state 0x1 ref 1 00000100:00100000:0.0:1789760765.003414:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746271627597 time 746318716985 state 0x0 ref 1 00000100:00100000:0.0:1789760765.003429:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760765.003486:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21446656, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xa807 00010000:00010000:0.0:1789760765.003496:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.003512:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760765.003517:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.003524:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.003538:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760765.004241:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760765.066742:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21446656 -> 21450752 00002000:00010000:0.0:1789760765.066752:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:0.0:1789760765.066754:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:0.0:1789760765.066767:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.066781:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.066791:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.066800:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.066806:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.066816:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.066837:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.066843:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00010000:00010000:0.0:1789760765.066847:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.066944:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63513us (63820us total) trans 4294972556 rc 0/0 00000100:00100000:0.0:1789760765.066954:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5266 00000100:00100000:0.0:1789760765.066963:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746271627597 time 746318716985 state 0x2 ref 0 00000100:00100000:3.0:1789760765.072027:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760765.072116:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304320 00000100:00100000:1.0:1789760765.072126:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304320 00000100:00100000:1.0:1789760765.072181:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746271627597 time 746318716985 state 0x0 ref 1 00000100:00100000:1.0:1789760765.072196:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746318977401 time 746318716985 state 0x1 ref 1 00000100:00100000:1.0:1789760765.072229:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746318977401 time 746387532003 state 0x0 ref 1 00000100:00100000:1.0:1789760765.072242:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760765.072293:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21450752, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:1.0:1789760765.072302:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.072318:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760765.072323:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.072329:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.072343:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.147814:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760765.147837:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538284928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760765.147868:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285056:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760765.147906:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760765.147960:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538284928 00000100:00100000:0.0:1789760765.147968:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538284928 00000100:00100000:0.0:1789760765.148029:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:0.0:1789760765.148040:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538284928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760765.148072:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3836739584 avail 3830448128 00000100:00100000:0.0:1789760765.148119:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538284928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760765.148130:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789760765.148186:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538284928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760765.148261:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760765.148337:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285056 00000100:00100000:0.0:1789760765.148341:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285056 00000100:00100000:0.0:1789760765.148372:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789760765.148379:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538285056:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760765.148390:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3836739584 avail 3830448128 00000100:00100000:0.0:1789760765.148440:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538285056:12345-0@lo:13:osp-pre-0-1.0 Request processed in 61us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760765.148447:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:2.0:1789760765.148572:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285056:0@lo:13:osp-pre-0-1.0 00000001:00080000:2.0:1789760765.150420:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00002000:00010000:1.0:1789760765.150573:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21450752 -> 21454848 00010000:00010000:1.0:1789760765.150587:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150600:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150608:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150629:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760765.150634:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150644:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150674:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.150681:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.150799:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 78554us (78777us total) trans 4294972557 rc 0/0 00000100:00100000:1.0:1789760765.150810:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5267 00000100:00100000:1.0:1789760765.150819:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746318977401 time 746387532003 state 0x2 ref 0 00000100:00100000:2.0:1789760765.158340:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760765.158438:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304448 00000100:00100000:1.0:1789760765.158448:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304448 00000100:00100000:1.0:1789760765.158501:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746318977401 time 746387532003 state 0x0 ref 1 00000100:00100000:1.0:1789760765.158517:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746387792419 time 746387532003 state 0x1 ref 1 00000100:00100000:1.0:1789760765.158547:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746387792419 time 746473850240 state 0x0 ref 1 00000100:00100000:1.0:1789760765.158563:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760765.158625:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21454848, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:1.0:1789760765.158634:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.158649:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760765.158653:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.158659:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.158671:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.256812:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21454848 -> 21458944 00010000:00010000:0.0:1789760765.256832:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256847:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256857:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256865:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.256870:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256880:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256907:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.256915:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8b75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.257011:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507304448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 98445us (98675us total) trans 4294972558 rc 0/0 00000100:00100000:0.0:1789760765.257021:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5268 00000100:00100000:0.0:1789760765.257030:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746387792419 time 746473850240 state 0x2 ref 0 00000001:00080000:3.0:1789760765.257174:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972558 is committed 00000100:00100000:2.0:1789760765.264081:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760765.264491:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304576 00000100:00100000:0.0:1789760765.264502:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304576 00000100:00100000:0.0:1789760765.264560:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746387792419 time 746473850240 state 0x0 ref 1 00000100:00100000:0.0:1789760765.264575:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746474110656 time 746473850240 state 0x1 ref 1 00000100:00100000:0.0:1789760765.264664:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746474110656 time 746579966143 state 0x0 ref 1 00000100:00100000:0.0:1789760765.264677:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760765.264756:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21458944, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:0.0:1789760765.264765:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.264780:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760765.264785:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.264792:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.264806:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.292305:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00002000:00010000:2.0:1789760765.293091:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21458944 -> 21463040 00010000:00010000:2.0:1789760765.293107:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293119:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293129:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293136:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760765.293140:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293153:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293171:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.293179:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8b7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.293279:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28598us (29203us total) trans 4294972559 rc 0/0 00000100:00100000:2.0:1789760765.293289:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5269 00000100:00100000:2.0:1789760765.293298:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746474110656 time 746579966143 state 0x2 ref 0 00000100:00100000:1.0:1789760765.296875:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.296966:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304704 00000100:00100000:2.0:1789760765.296975:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304704 00000100:00100000:2.0:1789760765.297029:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746474110656 time 746579966143 state 0x0 ref 1 00000100:00100000:2.0:1789760765.297044:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746580226559 time 746579966143 state 0x1 ref 1 00000100:00100000:2.0:1789760765.297080:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746580226559 time 746612383278 state 0x0 ref 1 00000100:00100000:2.0:1789760765.297092:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.297135:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21463040, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.297144:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.297158:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760765.297163:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.297169:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.297182:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.356299:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00002000:00010000:2.0:1789760765.356488:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:2.0:1789760765.356505:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356517:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356527:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356543:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760765.356548:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356557:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356577:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.356585:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8b83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.356877:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59780us (60004us total) trans 4294972560 rc 0/0 00000100:00100000:2.0:1789760765.356888:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5270 00000100:00100000:2.0:1789760765.356898:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746580226559 time 746612383278 state 0x2 ref 0 00000100:00100000:3.0:1789760765.362110:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760765.362255:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304832 00000100:00100000:1.0:1789760765.362264:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304832 00000100:00100000:1.0:1789760765.362322:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746580226559 time 746612383278 state 0x0 ref 1 00000100:00100000:1.0:1789760765.362410:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746612643694 time 746612383278 state 0x1 ref 1 00000100:00100000:1.0:1789760765.362441:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746612643694 time 746677743665 state 0x0 ref 1 00000100:00100000:1.0:1789760765.362455:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760765.362513:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21467136, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:1.0:1789760765.362523:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.363213:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760765.363218:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.363226:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.363243:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.402017:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00002000:00010000:1.0:1789760765.402184:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:1.0:1789760765.402200:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402213:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402223:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402231:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760765.402236:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402247:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402272:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.402281:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8b8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.402446:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507304832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39989us (40342us total) trans 4294972561 rc 0/0 00000100:00100000:1.0:1789760765.402456:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5271 00000100:00100000:1.0:1789760765.402465:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746612643694 time 746677743665 state 0x2 ref 0 00000100:00100000:2.0:1789760765.408045:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760765.408248:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507304960 00000100:00100000:0.0:1789760765.408258:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507304960 00000100:00100000:0.0:1789760765.408313:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746612643694 time 746677743665 state 0x0 ref 1 00000100:00100000:0.0:1789760765.408401:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746678004081 time 746677743665 state 0x1 ref 1 00000100:00100000:0.0:1789760765.408434:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746678004081 time 746723737524 state 0x0 ref 1 00000100:00100000:0.0:1789760765.408446:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760765.408493:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21471232, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:0.0:1789760765.408502:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.408516:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760765.408521:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.408590:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.408606:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.448476:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00002000:00010000:3.0:1789760765.448573:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21471232 -> 21475328 00010000:00010000:3.0:1789760765.448588:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448601:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448619:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448626:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760765.448631:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448652:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448673:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.448681:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8b91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.448785:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507304960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40337us (40746us total) trans 4294972562 rc 0/0 00000100:00100000:3.0:1789760765.448794:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5272 00000100:00100000:3.0:1789760765.448803:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746678004081 time 746723737524 state 0x2 ref 0 00000100:00100000:3.0:1789760765.452094:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760765.452352:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305088 00000100:00100000:0.0:1789760765.452362:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305088 00000100:00100000:0.0:1789760765.452411:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746678004081 time 746723737524 state 0x0 ref 1 00000100:00100000:0.0:1789760765.452425:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746723997940 time 746723737524 state 0x1 ref 1 00000100:00100000:0.0:1789760765.452447:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746723997940 time 746767751124 state 0x0 ref 1 00000100:00100000:0.0:1789760765.452459:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760765.452504:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21475328, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:0.0:1789760765.452512:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.452536:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760765.452541:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.452547:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.452561:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.482153:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00002000:00010000:0.0:1789760765.482229:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:0.0:1789760765.482244:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482264:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482273:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482281:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.482285:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482294:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482315:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.482323:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8b98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.482409:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29947us (30322us total) trans 4294972563 rc 0/0 00000100:00100000:0.0:1789760765.482420:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5273 00000100:00100000:0.0:1789760765.482429:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746723997940 time 746767751124 state 0x2 ref 0 00000100:00100000:1.0:1789760765.486033:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760765.486283:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305216 00000100:00100000:0.0:1789760765.486293:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305216 00000100:00100000:0.0:1789760765.486349:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746723997940 time 746767751124 state 0x0 ref 1 00000100:00100000:0.0:1789760765.486363:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746768011540 time 746767751124 state 0x1 ref 1 00000100:00100000:0.0:1789760765.486388:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746768011540 time 746801690903 state 0x0 ref 1 00000100:00100000:0.0:1789760765.486400:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760765.486445:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21479424, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:0.0:1789760765.486452:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.486464:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760765.486467:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.486471:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.486482:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.518046:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:1.0:1789760765.518230:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:1.0:1789760765.518244:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518258:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518267:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518274:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760765.518279:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518289:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518307:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.518315:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8b9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.518412:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32009us (32384us total) trans 4294972564 rc 0/0 00000100:00100000:1.0:1789760765.518423:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5274 00000100:00100000:1.0:1789760765.518432:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746768011540 time 746801690903 state 0x2 ref 0 00000100:00100000:2.0:1789760765.526435:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760765.526660:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305344 00000100:00100000:3.0:1789760765.526671:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305344 00000100:00100000:3.0:1789760765.526749:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746768011540 time 746801690903 state 0x0 ref 1 00000100:00100000:3.0:1789760765.526765:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746801951319 time 746801690903 state 0x1 ref 1 00000100:00100000:3.0:1789760765.526788:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746801951319 time 746842091893 state 0x0 ref 1 00000100:00100000:3.0:1789760765.526801:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760765.526953:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21483520, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:3.0:1789760765.526962:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.526977:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760765.526982:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.526988:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.527003:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.601512:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00002000:00010000:1.0:1789760765.601643:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:1.0:1789760765.601659:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601672:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601682:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601713:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760765.601718:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601727:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601746:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.601754:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8ba6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.601903:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75099us (75482us total) trans 4294972565 rc 0/0 00000100:00100000:1.0:1789760765.601915:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5275 00000100:00100000:1.0:1789760765.601925:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746801951319 time 746842091893 state 0x2 ref 0 00000100:00100000:0.0:1789760765.609822:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.610506:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305472 00000100:00100000:2.0:1789760765.610516:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305472 00000100:00100000:2.0:1789760765.610573:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746801951319 time 746842091893 state 0x0 ref 1 00000100:00100000:2.0:1789760765.610588:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746842352309 time 746842091893 state 0x1 ref 1 00000100:00100000:2.0:1789760765.610828:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746842352309 time 746926128448 state 0x0 ref 1 00000100:00100000:2.0:1789760765.610844:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.610892:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21487616, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.610901:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.610917:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760765.610922:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.610929:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.610944:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.656425:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:0.0:1789760765.656493:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21487616 -> 21491712 00010000:00010000:0.0:1789760765.656507:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656526:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656535:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656543:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.656547:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656555:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656574:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.656583:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8bad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.656730:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45846us (46875us total) trans 4294972566 rc 0/0 00000100:00100000:0.0:1789760765.656741:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5276 00000100:00100000:0.0:1789760765.656751:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746842352309 time 746926128448 state 0x2 ref 0 00000100:00100000:2.0:1789760765.660190:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538285184:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760765.660264:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760765.661313:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285184 00000100:00100000:1.0:1789760765.661324:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285184 00000100:00100000:1.0:1789760765.661380:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:1.0:1789760765.661390:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538285184:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760765.661428:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760765.661500:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538285184:12345-0@lo:41:osp-pre-1-0.0 Request processed in 112us (1236us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760765.661511:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789760765.662220:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760765.662280:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305600 00000100:00100000:1.0:1789760765.662285:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305600 00000100:00100000:1.0:1789760765.662321:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746842352309 time 746926128448 state 0x0 ref 1 00000100:00100000:1.0:1789760765.662335:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746926388864 time 746926128448 state 0x1 ref 1 00000100:00100000:1.0:1789760765.662366:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746926388864 time 746977669452 state 0x0 ref 1 00000100:00100000:1.0:1789760765.662378:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760765.662424:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21491712, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:1.0:1789760765.662433:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.662448:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760765.662452:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.662458:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.662472:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.663619:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538285184:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760765.710491:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00002000:00010000:2.0:1789760765.710749:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:2.0:1789760765.710764:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710776:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710784:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710791:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760765.710796:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710804:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710826:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.710834:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.711851:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49465us (49624us total) trans 4294972567 rc 0/0 00000100:00100000:2.0:1789760765.711865:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5277 00000100:00100000:2.0:1789760765.711875:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746926388864 time 746977669452 state 0x2 ref 0 00000100:00100000:2.0:1789760765.718918:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760765.719050:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305728 00000100:00100000:1.0:1789760765.719061:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305728 00000100:00100000:1.0:1789760765.719124:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746926388864 time 746977669452 state 0x0 ref 1 00000100:00100000:1.0:1789760765.719141:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746977929868 time 746977669452 state 0x1 ref 1 00000100:00100000:1.0:1789760765.719173:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746977929868 time 747034475960 state 0x0 ref 1 00000100:00100000:1.0:1789760765.719186:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760765.719231:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21495808, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:1.0:1789760765.719240:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.719256:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760765.719261:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.719266:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.719281:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.766265:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:3.0:1789760765.766343:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:3.0:1789760765.766357:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766370:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766378:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766384:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760765.766389:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766398:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766416:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.766423:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8bbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.766508:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507305728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47320us (47595us total) trans 4294972568 rc 0/0 00000100:00100000:3.0:1789760765.766518:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5278 00000100:00100000:3.0:1789760765.766528:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746977929868 time 747034475960 state 0x2 ref 0 00000100:00100000:0.0:1789760765.776947:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.777051:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305856 00000100:00100000:2.0:1789760765.777061:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305856 00000100:00100000:2.0:1789760765.777115:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 746977929868 time 747034475960 state 0x0 ref 1 00000100:00100000:2.0:1789760765.777131:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747034736376 time 747034475960 state 0x1 ref 1 00000100:00100000:2.0:1789760765.777160:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747034736376 time 747092463768 state 0x0 ref 1 00000100:00100000:2.0:1789760765.777172:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.777220:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21499904, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.777229:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.777244:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760765.777248:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.777255:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.777270:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.822559:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00002000:00010000:0.0:1789760765.822714:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21499904 -> 21504000 00010000:00010000:0.0:1789760765.822728:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822742:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822751:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822759:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.822763:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822773:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822792:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.822799:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8bc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.822880:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507305856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45705us (45941us total) trans 4294972569 rc 0/0 00000100:00100000:0.0:1789760765.822894:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5279 00000100:00100000:0.0:1789760765.822909:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747034736376 time 747092463768 state 0x2 ref 0 00000100:00100000:0.0:1789760765.826336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.826420:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507305984 00000100:00100000:2.0:1789760765.826430:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507305984 00000100:00100000:2.0:1789760765.826488:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747034736376 time 747092463768 state 0x0 ref 1 00000100:00100000:2.0:1789760765.826504:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747092724184 time 747092463768 state 0x1 ref 1 00000100:00100000:2.0:1789760765.826530:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747092724184 time 747141833428 state 0x0 ref 1 00000100:00100000:2.0:1789760765.826542:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507305984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.826587:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21504000, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.826596:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.826619:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760765.826623:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.826629:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.826651:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.854845:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760765.876565:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00002000:00010000:2.0:1789760765.876718:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21504000 -> 21508096 00010000:00010000:2.0:1789760765.876733:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876746:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876754:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876760:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760765.876765:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876773:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876793:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.876801:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.876900:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507305984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50355us (50568us total) trans 4294972570 rc 0/0 00000100:00100000:2.0:1789760765.876911:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5280 00000100:00100000:2.0:1789760765.876922:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747092724184 time 747141833428 state 0x2 ref 0 00000100:00100000:2.0:1789760765.880487:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760765.880587:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306112 00000100:00100000:3.0:1789760765.880597:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306112 00000100:00100000:3.0:1789760765.880775:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747092724184 time 747141833428 state 0x0 ref 1 00000100:00100000:3.0:1789760765.880793:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747142093844 time 747141833428 state 0x1 ref 1 00000100:00100000:3.0:1789760765.880863:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747142093844 time 747196166333 state 0x0 ref 1 00000100:00100000:3.0:1789760765.880877:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507306112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760765.880924:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21508096, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:3.0:1789760765.880933:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.880952:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760765.880956:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.880962:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.880976:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.930441:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00002000:00010000:3.0:1789760765.930587:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:3.0:1789760765.930611:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930625:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930633:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930728:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760765.930735:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930745:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930764:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.930773:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8bd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.930856:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507306112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49976us (50375us total) trans 4294972571 rc 0/0 00000100:00100000:3.0:1789760765.930866:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5281 00000100:00100000:3.0:1789760765.930876:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747142093844 time 747196166333 state 0x2 ref 0 00000100:00100000:0.0:1789760765.933755:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.933881:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306240 00000100:00100000:2.0:1789760765.933892:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306240 00000100:00100000:2.0:1789760765.933950:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747142093844 time 747196166333 state 0x0 ref 1 00000100:00100000:2.0:1789760765.933965:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747196426749 time 747196166333 state 0x1 ref 1 00000100:00100000:2.0:1789760765.933995:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747196426749 time 747249297918 state 0x0 ref 1 00000100:00100000:2.0:1789760765.934008:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.934054:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21512192, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.934063:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.934079:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760765.934084:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.934192:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.934210:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.966535:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00002000:00010000:0.0:1789760765.966724:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:0.0:1789760765.966741:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966754:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966763:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966770:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760765.966775:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966783:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966803:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.966815:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8bd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.966948:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32937us (33196us total) trans 4294972572 rc 0/0 00000100:00100000:0.0:1789760765.966958:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5282 00000100:00100000:0.0:1789760765.966967:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747196426749 time 747249297918 state 0x2 ref 0 00000100:00100000:0.0:1789760765.970984:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760765.971078:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306368 00000100:00100000:2.0:1789760765.971088:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306368 00000100:00100000:2.0:1789760765.971141:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747196426749 time 747249297918 state 0x0 ref 1 00000100:00100000:2.0:1789760765.971155:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747249558334 time 747249297918 state 0x1 ref 1 00000100:00100000:2.0:1789760765.971188:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747249558334 time 747286490765 state 0x0 ref 1 00000100:00100000:2.0:1789760765.971201:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760765.971249:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21516288, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:2.0:1789760765.971259:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.971273:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760765.971277:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.971284:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.971299:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.006201:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00002000:00010000:1.0:1789760766.006301:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:1.0:1789760766.006315:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006328:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006337:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006345:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760766.006350:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006358:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006377:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.006385:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8bde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.006548:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35344us (35568us total) trans 4294972573 rc 0/0 00000100:00100000:1.0:1789760766.006559:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5283 00000100:00100000:1.0:1789760766.006568:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747249558334 time 747286490765 state 0x2 ref 0 00000100:00100000:2.0:1789760766.010116:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760766.010196:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306496 00000100:00100000:3.0:1789760766.010205:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306496 00000100:00100000:3.0:1789760766.010257:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747249558334 time 747286490765 state 0x0 ref 1 00000100:00100000:3.0:1789760766.010273:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747286751181 time 747286490765 state 0x1 ref 1 00000100:00100000:3.0:1789760766.010321:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747286751181 time 747325609960 state 0x0 ref 1 00000100:00100000:3.0:1789760766.010335:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507306496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760766.010382:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21520384, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xa807 00010000:00010000:3.0:1789760766.010391:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.010406:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760766.010410:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.010417:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.010431:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.043179:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00002000:00010000:0.0:1789760766.043260:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21520384 -> 21524480 00002000:00010000:0.0:1789760766.043265:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:0.0:1789760766.043268:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:0.0:1789760766.043282:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043295:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043306:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043313:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.043318:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043333:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043353:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.043361:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8be5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.043486:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507306496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33148us (33374us total) trans 4294972574 rc 0/0 00000100:00100000:0.0:1789760766.043497:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5284 00000100:00100000:0.0:1789760766.043507:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747286751181 time 747325609960 state 0x2 ref 0 00000100:00100000:0.0:1789760766.051086:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.051176:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306624 00000100:00100000:2.0:1789760766.051186:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306624 00000100:00100000:2.0:1789760766.051240:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747286751181 time 747325609960 state 0x0 ref 1 00000100:00100000:2.0:1789760766.051255:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747325870376 time 747325609960 state 0x1 ref 1 00000100:00100000:2.0:1789760766.051285:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747325870376 time 747366588559 state 0x0 ref 1 00000100:00100000:2.0:1789760766.051297:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507306624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.051346:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21524480, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.051356:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.051372:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.051376:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.051383:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.051397:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.105936:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:0.0:1789760766.106033:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21524480 -> 21528576 00010000:00010000:0.0:1789760766.106048:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106062:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106072:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106085:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.106090:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106100:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106119:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.106127:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8bec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.106222:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507306624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54920us (55139us total) trans 4294972575 rc 0/0 00000100:00100000:0.0:1789760766.106232:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5285 00000100:00100000:0.0:1789760766.106242:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747325870376 time 747366588559 state 0x2 ref 0 00000100:00100000:2.0:1789760766.111851:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760766.112158:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306752 00000100:00100000:1.0:1789760766.112167:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306752 00000100:00100000:1.0:1789760766.112221:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747325870376 time 747366588559 state 0x0 ref 1 00000100:00100000:1.0:1789760766.112238:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747366848975 time 747366588559 state 0x1 ref 1 00000100:00100000:1.0:1789760766.112360:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747366848975 time 747427661226 state 0x0 ref 1 00000100:00100000:1.0:1789760766.112374:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507306752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760766.112424:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21528576, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:1.0:1789760766.112434:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.112449:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760766.112453:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.112459:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.112483:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.162983:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00002000:00010000:3.0:1789760766.163077:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:3.0:1789760766.163092:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163105:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163115:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163123:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760766.163127:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163136:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163156:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.163163:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b8bf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.163328:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507306752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50953us (51481us total) trans 4294972576 rc 0/0 00000100:00100000:3.0:1789760766.163338:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5286 00000100:00100000:3.0:1789760766.163347:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747366848975 time 747427661226 state 0x2 ref 0 00000100:00100000:2.0:1789760766.168405:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760766.168492:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507306880 00000100:00100000:0.0:1789760766.168511:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507306880 00000100:00100000:0.0:1789760766.168574:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747366848975 time 747427661226 state 0x0 ref 1 00000100:00100000:0.0:1789760766.168588:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747427921642 time 747427661226 state 0x1 ref 1 00000100:00100000:0.0:1789760766.168629:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747427921642 time 747483931910 state 0x0 ref 1 00000100:00100000:0.0:1789760766.168642:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760766.168733:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21532672, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:0.0:1789760766.168742:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.168758:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760766.168762:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.168769:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.168783:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.173392:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760766.178417:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108016d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285312:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760766.178485:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.178626:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285312 00000100:00100000:1.0:1789760766.178634:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285312 00000100:00100000:1.0:1789760766.178711:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1789760766.178719:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538285312:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760766.178753:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760766.178799:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538285312:12345-0@lo:41:osp-pre-0-1.0 Request processed in 80us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.178806:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:3.0:1789760766.178997:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108016d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285312:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760766.211086:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00002000:00010000:0.0:1789760766.211172:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21532672 -> 21536768 00010000:00010000:0.0:1789760766.211252:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211264:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211272:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211279:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.211284:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211293:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211311:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.211319:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b8bfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.211441:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507306880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42796us (43040us total) trans 4294972577 rc 0/0 00000100:00100000:0.0:1789760766.211452:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5287 00000100:00100000:0.0:1789760766.211462:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747427921642 time 747483931910 state 0x2 ref 0 00000100:00100000:0.0:1789760766.217386:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.217945:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307008 00000100:00100000:2.0:1789760766.217958:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307008 00000100:00100000:2.0:1789760766.218011:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747427921642 time 747483931910 state 0x0 ref 1 00000100:00100000:2.0:1789760766.218026:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747484192326 time 747483931910 state 0x1 ref 1 00000100:00100000:2.0:1789760766.218055:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747484192326 time 747533358653 state 0x0 ref 1 00000100:00100000:2.0:1789760766.218067:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507307008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.218113:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21536768, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.218121:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.218137:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.218141:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.218147:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.218161:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.267176:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00002000:00010000:2.0:1789760766.267216:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21536768 -> 21540864 00010000:00010000:2.0:1789760766.267229:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267242:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267253:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267260:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.267265:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267275:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267294:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.267302:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.267443:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507307008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49371us (50061us total) trans 4294972578 rc 0/0 00000100:00100000:2.0:1789760766.267454:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5288 00000100:00100000:2.0:1789760766.267464:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747484192326 time 747533358653 state 0x2 ref 0 00000100:00100000:2.0:1789760766.273481:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760766.273652:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307136 00000100:00100000:3.0:1789760766.273663:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307136 00000100:00100000:3.0:1789760766.273750:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747484192326 time 747533358653 state 0x0 ref 1 00000100:00100000:3.0:1789760766.273765:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747533619069 time 747533358653 state 0x1 ref 1 00000100:00100000:3.0:1789760766.273794:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747533619069 time 747589097265 state 0x0 ref 1 00000100:00100000:3.0:1789760766.273806:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507307136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760766.273850:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21540864, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:3.0:1789760766.273858:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.273873:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760766.273877:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.273883:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.273898:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.303495:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:2.0:1789760766.303951:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:2.0:1789760766.303966:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.303980:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.303990:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.304003:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.304008:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.304016:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.304036:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.304043:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8c08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.304435:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507307136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30575us (30907us total) trans 4294972579 rc 0/0 00000100:00100000:2.0:1789760766.304447:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5289 00000100:00100000:2.0:1789760766.304457:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747533619069 time 747589097265 state 0x2 ref 0 00000100:00100000:0.0:1789760766.311057:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760766.311339:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307264 00000100:00100000:3.0:1789760766.311349:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307264 00000100:00100000:3.0:1789760766.311414:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747533619069 time 747589097265 state 0x0 ref 1 00000100:00100000:3.0:1789760766.311431:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747589357681 time 747589097265 state 0x1 ref 1 00000100:00100000:3.0:1789760766.311504:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747589357681 time 747626805405 state 0x0 ref 1 00000100:00100000:3.0:1789760766.311517:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507307264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760766.311562:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21544960, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:3.0:1789760766.311571:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.311586:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760766.311591:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.311598:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.311622:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.341827:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21544960 -> 21549056 00010000:00010000:3.0:1789760766.341847:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341860:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341869:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341877:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760766.341882:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341890:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341910:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.341920:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.342013:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507307264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30493us (30961us total) trans 4294972580 rc 0/0 00000100:00100000:3.0:1789760766.342023:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5290 00000100:00100000:3.0:1789760766.342033:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747589357681 time 747626805405 state 0x2 ref 0 00000001:00080000:2.0:1789760766.342943:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00000100:00100000:1.0:1789760766.346495:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.346583:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307392 00000100:00100000:2.0:1789760766.346593:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307392 00000100:00100000:2.0:1789760766.346657:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747589357681 time 747626805405 state 0x0 ref 1 00000100:00100000:2.0:1789760766.346683:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747627065821 time 747626805405 state 0x1 ref 1 00000100:00100000:1.0:1789760766.346887:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747627065821 time 747662187857 state 0x0 ref 1 00000100:00100000:1.0:1789760766.346902:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760766.346950:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21549056, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:1.0:1789760766.346959:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.346975:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760766.346980:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.346986:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.347000:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.380525:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00002000:00010000:1.0:1789760766.381323:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21549056 -> 21553152 00010000:00010000:1.0:1789760766.381338:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381351:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381359:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381365:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760766.381370:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381378:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381396:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381404:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8c16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.381490:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34586us (34998us total) trans 4294972581 rc 0/0 00000100:00100000:1.0:1789760766.381500:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5291 00000100:00100000:1.0:1789760766.381509:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747627065821 time 747662187857 state 0x2 ref 0 00000100:00100000:0.0:1789760766.388615:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.388755:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307520 00000100:00100000:2.0:1789760766.388765:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307520 00000100:00100000:2.0:1789760766.388820:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747627065821 time 747662187857 state 0x0 ref 1 00000100:00100000:2.0:1789760766.388835:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747662448273 time 747662187857 state 0x1 ref 1 00000100:00100000:2.0:1789760766.388899:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747662448273 time 747704202327 state 0x0 ref 1 00000100:00100000:2.0:1789760766.388911:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507307520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.388957:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21553152, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.388967:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.388983:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.388988:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.388994:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.389008:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.419524:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00002000:00010000:3.0:1789760766.419593:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21553152 -> 21557248 00010000:00010000:3.0:1789760766.419618:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419631:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419648:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419656:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760766.419661:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419670:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419688:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.419731:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.419823:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507307520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30909us (31223us total) trans 4294972582 rc 0/0 00000100:00100000:3.0:1789760766.419834:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5292 00000100:00100000:3.0:1789760766.419843:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747662448273 time 747704202327 state 0x2 ref 0 00000100:00100000:0.0:1789760766.423427:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760766.424152:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307648 00000100:00100000:3.0:1789760766.424163:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307648 00000100:00100000:3.0:1789760766.424285:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747662448273 time 747704202327 state 0x0 ref 1 00000100:00100000:3.0:1789760766.424301:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747704462743 time 747704202327 state 0x1 ref 1 00000100:00100000:3.0:1789760766.424378:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747704462743 time 747739680591 state 0x0 ref 1 00000100:00100000:3.0:1789760766.424393:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760766.424440:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21557248, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:3.0:1789760766.424450:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.424466:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760766.424471:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.424477:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.424490:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.464346:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:0.0:1789760766.464454:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:0.0:1789760766.464470:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464482:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464495:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464501:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.464505:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464514:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464599:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.464616:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8c24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.464879:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40485us (41457us total) trans 4294972583 rc 0/0 00000100:00100000:0.0:1789760766.464889:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5293 00000100:00100000:0.0:1789760766.464899:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747704462743 time 747739680591 state 0x2 ref 0 00000100:00100000:1.0:1789760766.476617:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.476756:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307776 00000100:00100000:2.0:1789760766.476766:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307776 00000100:00100000:2.0:1789760766.476819:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747704462743 time 747739680591 state 0x0 ref 1 00000100:00100000:2.0:1789760766.476832:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747739941007 time 747739680591 state 0x1 ref 1 00000100:00100000:2.0:1789760766.476907:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747739941007 time 747792209501 state 0x0 ref 1 00000100:00100000:2.0:1789760766.476920:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507307776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.476980:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21561344, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.476990:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.477006:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.477011:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.477017:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.477030:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.516011:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00002000:00010000:2.0:1789760766.516395:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:2.0:1789760766.516411:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516424:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516432:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516439:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.516444:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516453:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516473:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.516481:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.516618:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507307776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39695us (40086us total) trans 4294972584 rc 0/0 00000100:00100000:2.0:1789760766.516627:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5294 00000100:00100000:2.0:1789760766.516641:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747739941007 time 747792209501 state 0x2 ref 0 00000100:00100000:1.0:1789760766.522126:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760766.522236:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507307904 00000100:00100000:3.0:1789760766.522246:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507307904 00000100:00100000:3.0:1789760766.522365:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747739941007 time 747792209501 state 0x0 ref 1 00000100:00100000:3.0:1789760766.522383:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747792469917 time 747792209501 state 0x1 ref 1 00000100:00100000:3.0:1789760766.522450:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747792469917 time 747837752762 state 0x0 ref 1 00000100:00100000:3.0:1789760766.522463:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760766.522512:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21565440, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:3.0:1789760766.522522:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.522551:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760766.522556:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.522562:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.522577:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.554914:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:2.0:1789760766.555060:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21565440 -> 21569536 00010000:00010000:2.0:1789760766.555075:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555088:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555096:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555107:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.555112:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555120:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555138:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.555145:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8c32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.555235:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507307904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32770us (33114us total) trans 4294972585 rc 0/0 00000100:00100000:2.0:1789760766.555245:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5295 00000100:00100000:2.0:1789760766.555254:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747792469917 time 747837752762 state 0x2 ref 0 00000100:00100000:3.0:1789760766.559114:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760766.559357:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308032 00000100:00100000:1.0:1789760766.559368:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308032 00000100:00100000:1.0:1789760766.559421:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747792469917 time 747837752762 state 0x0 ref 1 00000100:00100000:1.0:1789760766.559435:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747838013178 time 747837752762 state 0x1 ref 1 00000100:00100000:1.0:1789760766.559465:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747838013178 time 747874768068 state 0x0 ref 1 00000100:00100000:1.0:1789760766.559476:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507308032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760766.559519:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21569536, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:1.0:1789760766.559536:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.559551:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760766.559555:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.559563:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.559578:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.593299:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00002000:00010000:2.0:1789760766.593433:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21569536 -> 21573632 00010000:00010000:2.0:1789760766.593447:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593460:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593474:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593482:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.593487:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593495:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593514:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.593522:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8c39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.593797:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507308032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34317us (34687us total) trans 4294972586 rc 0/0 00000100:00100000:2.0:1789760766.593808:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5296 00000100:00100000:2.0:1789760766.593818:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747838013178 time 747874768068 state 0x2 ref 0 00000100:00100000:1.0:1789760766.597523:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.597808:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308160 00000100:00100000:2.0:1789760766.597818:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308160 00000100:00100000:2.0:1789760766.597869:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747838013178 time 747874768068 state 0x0 ref 1 00000100:00100000:2.0:1789760766.597883:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747875028484 time 747874768068 state 0x1 ref 1 00000100:00100000:2.0:1789760766.597904:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747875028484 time 747913208016 state 0x0 ref 1 00000100:00100000:2.0:1789760766.597914:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507308160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.597955:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21573632, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.597963:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597978:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.597982:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.597988:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.598001:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.639208:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00002000:00010000:2.0:1789760766.639276:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21573632 -> 21577728 00010000:00010000:2.0:1789760766.639292:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639305:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639314:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639322:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.639326:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639335:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639354:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.639361:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.639509:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507308160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41590us (41990us total) trans 4294972587 rc 0/0 00000100:00100000:2.0:1789760766.639519:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5297 00000100:00100000:2.0:1789760766.639529:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747875028484 time 747913208016 state 0x2 ref 0 00000100:00100000:0.0:1789760766.648658:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.649011:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308288 00000100:00100000:2.0:1789760766.649021:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308288 00000100:00100000:2.0:1789760766.649079:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747875028484 time 747913208016 state 0x0 ref 1 00000100:00100000:2.0:1789760766.649095:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747913468432 time 747913208016 state 0x1 ref 1 00000100:00100000:2.0:1789760766.649129:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747913468432 time 747964431858 state 0x0 ref 1 00000100:00100000:2.0:1789760766.649141:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507308288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.649367:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21577728, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.649378:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.649393:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.649398:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.649404:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.649419:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.677087:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:0.0:1789760766.677300:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:0.0:1789760766.677317:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677330:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677340:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677348:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.677352:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677360:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677380:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.677388:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b8c47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.677545:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507308288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28400us (28893us total) trans 4294972588 rc 0/0 00000100:00100000:0.0:1789760766.677557:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5298 00000100:00100000:0.0:1789760766.677567:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747913468432 time 747964431858 state 0x2 ref 0 00000100:00100000:3.0:1789760766.684010:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.684730:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308416 00000100:00100000:2.0:1789760766.684741:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308416 00000100:00100000:2.0:1789760766.684797:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747913468432 time 747964431858 state 0x0 ref 1 00000100:00100000:2.0:1789760766.684812:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747964692274 time 747964431858 state 0x1 ref 1 00000100:00100000:2.0:1789760766.684832:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747964692274 time 748000135427 state 0x0 ref 1 00000100:00100000:2.0:1789760766.684843:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507308416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.684890:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21581824, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.684898:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.684914:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760766.684918:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.684925:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.684939:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.729411:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972589 is committed 00002000:00010000:1.0:1789760766.729516:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:1.0:1789760766.729530:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729543:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729552:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729559:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760766.729563:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729572:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729595:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.729618:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8c4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.729753:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507308416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44906us (45747us total) trans 4294972589 rc 0/0 00000100:00100000:1.0:1789760766.729765:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5299 00000100:00100000:1.0:1789760766.729776:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747964692274 time 748000135427 state 0x2 ref 0 00000100:00100000:1.0:1789760766.733976:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.734212:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308544 00000100:00100000:2.0:1789760766.734221:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308544 00000100:00100000:2.0:1789760766.734273:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 747964692274 time 748000135427 state 0x0 ref 1 00000100:00100000:2.0:1789760766.734288:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748000395843 time 748000135427 state 0x1 ref 1 00000100:00100000:2.0:1789760766.734318:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748000395843 time 748049621031 state 0x0 ref 1 00000100:00100000:2.0:1789760766.734330:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507308544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.734374:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21585920, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.734383:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.734400:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760766.734405:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.734411:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.734426:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.774930:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00002000:00010000:2.0:1789760766.775255:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:2.0:1789760766.775270:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775283:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775291:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775299:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.775303:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775311:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775329:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.775337:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.775427:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507308544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41094us (41456us total) trans 4294972590 rc 0/0 00000100:00100000:2.0:1789760766.775436:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5300 00000100:00100000:2.0:1789760766.775445:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748000395843 time 748049621031 state 0x2 ref 0 00000100:00100000:0.0:1789760766.778896:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.778972:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308672 00000100:00100000:2.0:1789760766.778983:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308672 00000100:00100000:2.0:1789760766.779037:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748000395843 time 748049621031 state 0x0 ref 1 00000100:00100000:2.0:1789760766.779173:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748049881447 time 748049621031 state 0x1 ref 1 00000100:00100000:2.0:1789760766.779203:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748049881447 time 748094506543 state 0x0 ref 1 00000100:00100000:2.0:1789760766.779216:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507308672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.779266:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21590016, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.779274:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.779292:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760766.779297:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.779304:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.779319:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.810564:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:2.0:1789760766.810855:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21590016 -> 21594112 00010000:00010000:2.0:1789760766.810871:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810884:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810893:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810900:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.810905:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810913:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810932:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810940:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b8c5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.811073:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507308672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31854us (32180us total) trans 4294972591 rc 0/0 00000100:00100000:2.0:1789760766.811082:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5301 00000100:00100000:2.0:1789760766.811092:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748049881447 time 748094506543 state 0x2 ref 0 00000100:00100000:3.0:1789760766.815253:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760766.815555:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308800 00000100:00100000:1.0:1789760766.815565:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308800 00000100:00100000:1.0:1789760766.815629:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748049881447 time 748094506543 state 0x0 ref 1 00000100:00100000:1.0:1789760766.815653:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748094766959 time 748094506543 state 0x1 ref 1 00000100:00100000:1.0:1789760766.815680:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748094766959 time 748130983534 state 0x0 ref 1 00000100:00100000:1.0:1789760766.815721:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507308800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760766.815770:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21594112, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:1.0:1789760766.815779:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.815796:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760766.815800:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.815807:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.815822:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.852065:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:1.0:1789760766.852085:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852097:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852105:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852114:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760766.852118:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852127:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852146:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.852153:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b8c63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.852248:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507308800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36553us (37000us total) trans 4294972592 rc 0/0 00000100:00100000:1.0:1789760766.852259:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5302 00000100:00100000:1.0:1789760766.852269:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748094766959 time 748130983534 state 0x2 ref 0 00000001:00080000:0.0:1789760766.853595:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00000100:00100000:1.0:1789760766.855858:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.855952:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507308928 00000100:00100000:2.0:1789760766.855962:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507308928 00000100:00100000:2.0:1789760766.858259:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748094766959 time 748130983534 state 0x0 ref 1 00000100:00100000:2.0:1789760766.858279:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748131243950 time 748130983534 state 0x1 ref 1 00000100:00100000:2.0:1789760766.858320:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748131243950 time 748173622752 state 0x0 ref 1 00000100:00100000:2.0:1789760766.858333:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507308928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.858383:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21598208, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.858392:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.858408:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760766.858413:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.858419:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.858434:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.875791:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760766.887321:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00002000:00010000:0.0:1789760766.887483:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:0.0:1789760766.887500:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887512:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887520:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887527:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760766.887531:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887539:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887557:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.887564:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8c6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.889631:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507308928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31289us (33771us total) trans 4294972593 rc 0/0 00000100:00100000:0.0:1789760766.889658:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5303 00000100:00100000:0.0:1789760766.889668:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748131243950 time 748173622752 state 0x2 ref 0 00000100:00100000:0.0:1789760766.894098:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760766.894275:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309056 00000100:00100000:1.0:1789760766.894286:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309056 00000100:00100000:1.0:1789760766.894339:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748131243950 time 748173622752 state 0x0 ref 1 00000100:00100000:1.0:1789760766.894354:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748173883168 time 748173622752 state 0x1 ref 1 00000100:00100000:1.0:1789760766.894385:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748173883168 time 748209688790 state 0x0 ref 1 00000100:00100000:1.0:1789760766.894398:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507309056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760766.894445:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21602304, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:1.0:1789760766.894455:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.894471:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760766.894475:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.894482:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.894502:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.946432:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:1.0:1789760766.946452:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946465:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946474:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946482:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760766.946486:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946496:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946515:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.946523:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8c71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.946630:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507309056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52230us (52538us total) trans 4294972594 rc 0/0 00000100:00100000:1.0:1789760766.946648:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5304 00000100:00100000:1.0:1789760766.946657:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748173883168 time 748209688790 state 0x2 ref 0 00000001:00080000:2.0:1789760766.947173:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972594 is committed 00000100:00100000:1.0:1789760766.949912:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760766.949987:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309184 00000100:00100000:2.0:1789760766.949996:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309184 00000100:00100000:2.0:1789760766.950041:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748173883168 time 748209688790 state 0x0 ref 1 00000100:00100000:2.0:1789760766.950061:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748209949206 time 748209688790 state 0x1 ref 1 00000100:00100000:2.0:1789760766.950092:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748209949206 time 748265394895 state 0x0 ref 1 00000100:00100000:2.0:1789760766.950103:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507309184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760766.950155:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21606400, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760766.950164:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.950179:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760766.950184:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.950192:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.950206:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.993557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972595 is committed 00002000:00010000:2.0:1789760766.993638:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:2.0:1789760766.993654:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993667:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993676:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993683:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760766.993688:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993724:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993744:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.993753:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.994031:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507309184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43925us (44125us total) trans 4294972595 rc 0/0 00000100:00100000:2.0:1789760766.994042:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5305 00000100:00100000:2.0:1789760766.994051:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748209949206 time 748265394895 state 0x2 ref 0 00000100:00100000:0.0:1789760767.003676:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.004125:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309312 00000100:00100000:2.0:1789760767.004135:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309312 00000100:00100000:2.0:1789760767.004297:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748209949206 time 748265394895 state 0x0 ref 1 00000100:00100000:2.0:1789760767.004314:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748265655311 time 748265394895 state 0x1 ref 1 00000100:00100000:2.0:1789760767.004347:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748265655311 time 748319650604 state 0x0 ref 1 00000100:00100000:2.0:1789760767.004360:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507309312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.004517:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21610496, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xa807 00010000:00010000:2.0:1789760767.004527:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.004544:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.004548:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.004554:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.004568:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760767.005011:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760767.039017:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00002000:00010000:0.0:1789760767.039186:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21610496 -> 21614592 00002000:00010000:0.0:1789760767.039191:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:0.0:1789760767.039193:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00010000:00010000:0.0:1789760767.039207:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039219:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039228:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039236:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.039240:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039248:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039269:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.039281:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8c7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.039376:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507309312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35014us (35704us total) trans 4294972596 rc 0/0 00000100:00100000:0.0:1789760767.039452:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5306 00000100:00100000:0.0:1789760767.039463:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748265655311 time 748319650604 state 0x2 ref 0 00000100:00100000:3.0:1789760767.042860:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760767.043611:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309440 00000100:00100000:1.0:1789760767.043622:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309440 00000100:00100000:1.0:1789760767.043725:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748265655311 time 748319650604 state 0x0 ref 1 00000100:00100000:1.0:1789760767.043742:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748319911020 time 748319650604 state 0x1 ref 1 00000100:00100000:1.0:1789760767.043765:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748319911020 time 748359068380 state 0x0 ref 1 00000100:00100000:1.0:1789760767.043778:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760767.043836:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21614592, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:1.0:1789760767.043845:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.043860:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760767.043865:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.043871:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.043886:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.081292:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:1.0:1789760767.081314:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081327:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081335:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081343:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.081347:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081357:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081376:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.081397:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.081491:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507309440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37709us (38636us total) trans 4294972597 rc 0/0 00000100:00100000:1.0:1789760767.081504:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5307 00000100:00100000:1.0:1789760767.081514:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748319911020 time 748359068380 state 0x2 ref 0 00000001:00080000:3.0:1789760767.084119:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00000100:00100000:1.0:1789760767.085227:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760767.085366:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309568 00000100:00100000:3.0:1789760767.085375:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309568 00000100:00100000:3.0:1789760767.085440:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748319911020 time 748359068380 state 0x0 ref 1 00000100:00100000:3.0:1789760767.085456:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748359328796 time 748359068380 state 0x1 ref 1 00000100:00100000:3.0:1789760767.085489:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748359328796 time 748400791519 state 0x0 ref 1 00000100:00100000:3.0:1789760767.085501:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760767.085547:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21618688, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:3.0:1789760767.085556:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.085571:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760767.085576:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.085582:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.085606:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.127557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00002000:00010000:3.0:1789760767.127749:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:3.0:1789760767.127765:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127782:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127856:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127863:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760767.127868:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127878:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127898:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.127906:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8c8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.128004:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42501us (42780us total) trans 4294972598 rc 0/0 00000100:00100000:3.0:1789760767.128019:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5308 00000100:00100000:3.0:1789760767.128028:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748359328796 time 748400791519 state 0x2 ref 0 00000100:00100000:0.0:1789760767.132123:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.132407:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309696 00000100:00100000:2.0:1789760767.132473:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309696 00000100:00100000:2.0:1789760767.132528:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748359328796 time 748400791519 state 0x0 ref 1 00000100:00100000:2.0:1789760767.132549:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748401051935 time 748400791519 state 0x1 ref 1 00000100:00100000:2.0:1789760767.132582:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748401051935 time 748447885381 state 0x0 ref 1 00000100:00100000:2.0:1789760767.132594:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.132654:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21622784, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.132663:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.132679:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.132683:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.132689:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.132739:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.164405:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21622784 -> 21626880 00010000:00010000:2.0:1789760767.164425:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164439:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164448:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164454:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760767.164459:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164468:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164487:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.164494:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8c94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.164593:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507309696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31997us (32474us total) trans 4294972599 rc 0/0 00000100:00100000:2.0:1789760767.164612:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5309 00000100:00100000:2.0:1789760767.164622:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748401051935 time 748447885381 state 0x2 ref 0 00000001:00080000:1.0:1789760767.164862:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00000100:00100000:2.0:1789760767.168569:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760767.169000:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309824 00000100:00100000:1.0:1789760767.169010:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309824 00000100:00100000:1.0:1789760767.169078:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748401051935 time 748447885381 state 0x0 ref 1 00000100:00100000:1.0:1789760767.169093:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748448145797 time 748447885381 state 0x1 ref 1 00000100:00100000:1.0:1789760767.169126:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748448145797 time 748484429007 state 0x0 ref 1 00000100:00100000:1.0:1789760767.169139:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507309824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760767.169184:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21626880, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:1.0:1789760767.169192:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.169207:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760767.169211:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.169216:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.169229:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.198999:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:1.0:1789760767.199018:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199032:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199040:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199047:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.199052:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199061:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199080:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.199087:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b8c9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.199178:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507309824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30037us (30614us total) trans 4294972600 rc 0/0 00000100:00100000:1.0:1789760767.200237:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5310 00000100:00100000:1.0:1789760767.200254:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748448145797 time 748484429007 state 0x2 ref 0 00000001:00080000:2.0:1789760767.202460:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00000100:00100000:3.0:1789760767.205381:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.205975:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507309952 00000100:00100000:2.0:1789760767.205985:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507309952 00000100:00100000:0.0:1789760767.206230:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748448145797 time 748484429007 state 0x0 ref 1 00000100:00100000:0.0:1789760767.206247:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748484689423 time 748484429007 state 0x1 ref 1 00000100:00100000:0.0:1789760767.206332:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748484689423 time 748521633552 state 0x0 ref 1 00000100:00100000:0.0:1789760767.206345:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.206394:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21630976, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.206403:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.206419:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760767.206424:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.206430:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.206450:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.235407:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00002000:00010000:0.0:1789760767.235567:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:0.0:1789760767.235582:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235606:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235616:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235623:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.235628:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235765:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235785:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.235793:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ca2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.235922:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507309952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29575us (30547us total) trans 4294972601 rc 0/0 00000100:00100000:0.0:1789760767.235932:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5311 00000100:00100000:0.0:1789760767.235940:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748484689423 time 748521633552 state 0x2 ref 0 00000100:00100000:3.0:1789760767.239849:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.239924:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310080 00000100:00100000:2.0:1789760767.239935:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310080 00000100:00100000:2.0:1789760767.239987:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748484689423 time 748521633552 state 0x0 ref 1 00000100:00100000:2.0:1789760767.240002:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748521893968 time 748521633552 state 0x1 ref 1 00000100:00100000:2.0:1789760767.240034:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748521893968 time 748555336947 state 0x0 ref 1 00000100:00100000:2.0:1789760767.240054:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.240208:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21635072, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.240216:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.240231:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.240236:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.240243:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.240258:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.277026:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00002000:00010000:2.0:1789760767.277234:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:2.0:1789760767.277250:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277264:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277273:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277280:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760767.277284:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277294:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277314:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.277321:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8ca9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.277412:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+6:2416:x1876699507310080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37355us (37568us total) trans 4294972602 rc 0/0 00000100:00100000:2.0:1789760767.277423:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5312 00000100:00100000:2.0:1789760767.277432:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748521893968 time 748555336947 state 0x2 ref 0 00000100:00100000:1.0:1789760767.281315:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.281408:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310208 00000100:00100000:2.0:1789760767.281418:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310208 00000100:00100000:2.0:1789760767.281471:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748521893968 time 748555336947 state 0x0 ref 1 00000100:00100000:2.0:1789760767.281486:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748555597363 time 748555336947 state 0x1 ref 1 00000100:00100000:2.0:1789760767.281549:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748555597363 time 748596851988 state 0x0 ref 1 00000100:00100000:2.0:1789760767.281562:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507310208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.281618:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21639168, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.281628:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.281653:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.281658:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.281664:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.281679:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.323444:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00002000:00010000:3.0:1789760767.323522:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:3.0:1789760767.323537:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323550:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323557:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323562:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760767.323565:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323571:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323585:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.323590:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8cb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.323854:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507310208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42289us (42542us total) trans 4294972603 rc 0/0 00000100:00100000:3.0:1789760767.323873:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5313 00000100:00100000:3.0:1789760767.323883:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748555597363 time 748596851988 state 0x2 ref 0 00000100:00100000:2.0:1789760767.326232:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.326312:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310336 00000100:00100000:0.0:1789760767.326322:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310336 00000100:00100000:0.0:1789760767.326369:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748555597363 time 748596851988 state 0x0 ref 1 00000100:00100000:0.0:1789760767.326380:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748597112404 time 748596851988 state 0x1 ref 1 00000100:00100000:0.0:1789760767.326764:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748597112404 time 748642066120 state 0x0 ref 1 00000100:00100000:0.0:1789760767.326778:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507310336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.326834:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21643264, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.326843:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.326859:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760767.326863:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.326868:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.326878:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.378057:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00002000:00010000:0.0:1789760767.378128:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21643264 -> 21647360 00010000:00010000:0.0:1789760767.378143:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378156:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378165:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378173:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.378178:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378194:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378219:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.378227:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.378333:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507310336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51554us (52104us total) trans 4294972604 rc 0/0 00000100:00100000:0.0:1789760767.378348:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5314 00000100:00100000:0.0:1789760767.378357:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748597112404 time 748642066120 state 0x2 ref 0 00000100:00100000:1.0:1789760767.383326:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.383420:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310464 00000100:00100000:2.0:1789760767.383430:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310464 00000100:00100000:2.0:1789760767.383484:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748597112404 time 748642066120 state 0x0 ref 1 00000100:00100000:2.0:1789760767.383499:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748642326536 time 748642066120 state 0x1 ref 1 00000100:00100000:2.0:1789760767.383526:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748642326536 time 748698830296 state 0x0 ref 1 00000100:00100000:2.0:1789760767.383539:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.383583:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21647360, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.383591:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.383616:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.383620:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.383626:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.383657:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.442479:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00002000:00010000:2.0:1789760767.442574:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:2.0:1789760767.442589:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442617:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442625:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442732:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760767.442737:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442747:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442767:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.442774:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.443000:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59457us (59677us total) trans 4294972605 rc 0/0 00000100:00100000:2.0:1789760767.443010:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5315 00000100:00100000:2.0:1789760767.443019:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748642326536 time 748698830296 state 0x2 ref 0 00000100:00100000:3.0:1789760767.451086:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.451150:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310592 00000100:00100000:2.0:1789760767.451164:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310592 00000100:00100000:2.0:1789760767.451218:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748642326536 time 748698830296 state 0x0 ref 1 00000100:00100000:2.0:1789760767.451233:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748699090712 time 748698830296 state 0x1 ref 1 00000100:00100000:2.0:1789760767.451264:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748699090712 time 748766567434 state 0x0 ref 1 00000100:00100000:2.0:1789760767.451276:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.451327:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21651456, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.451336:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.451355:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.451359:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.451365:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.451379:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.533248:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00002000:00010000:0.0:1789760767.533384:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21651456 -> 21655552 00010000:00010000:0.0:1789760767.533400:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533413:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533422:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533431:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.533436:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533446:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533466:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.533475:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8cc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.533570:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82292us (82488us total) trans 4294972606 rc 0/0 00000100:00100000:0.0:1789760767.533581:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5316 00000100:00100000:0.0:1789760767.533591:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748699090712 time 748766567434 state 0x2 ref 0 00000100:00100000:3.0:1789760767.540617:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.540721:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310720 00000100:00100000:0.0:1789760767.540732:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310720 00000100:00100000:0.0:1789760767.540786:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748699090712 time 748766567434 state 0x0 ref 1 00000100:00100000:0.0:1789760767.540801:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748766827850 time 748766567434 state 0x1 ref 1 00000100:00100000:0.0:1789760767.540837:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748766827850 time 748856140191 state 0x0 ref 1 00000100:00100000:0.0:1789760767.540850:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507310720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.540911:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21655552, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.540922:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.540939:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760767.540945:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.540952:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.540967:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.594965:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21655552 -> 21659648 00010000:00010000:0.0:1789760767.594984:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.594997:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.595007:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.595015:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.595020:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.595030:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.595047:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.595055:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8ccc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.595139:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507310720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54287us (54527us total) trans 4294972607 rc 0/0 00000100:00100000:0.0:1789760767.595154:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5317 00000100:00100000:0.0:1789760767.595163:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748766827850 time 748856140191 state 0x2 ref 0 00000001:00080000:2.0:1789760767.596151:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972607 is committed 00000100:00100000:1.0:1789760767.608055:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.608150:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310848 00000100:00100000:2.0:1789760767.608161:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310848 00000100:00100000:2.0:1789760767.608287:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748766827850 time 748856140191 state 0x0 ref 1 00000100:00100000:2.0:1789760767.608302:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748856400607 time 748856140191 state 0x1 ref 1 00000100:00100000:2.0:1789760767.608473:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748856400607 time 748923775227 state 0x0 ref 1 00000100:00100000:2.0:1789760767.608488:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.608535:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21659648, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.608545:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.608560:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760767.608565:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.608571:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.608586:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.644662:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:0.0:1789760767.644681:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644722:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644731:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644739:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760767.644745:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644754:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644774:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.644781:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8cd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.644871:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507310848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36382us (36822us total) trans 4294972608 rc 0/0 00000100:00100000:0.0:1789760767.644882:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5318 00000100:00100000:0.0:1789760767.644891:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748856400607 time 748923775227 state 0x2 ref 0 00000001:00080000:2.0:1789760767.645253:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972608 is committed 00000100:00100000:3.0:1789760767.654452:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.654559:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507310976 00000100:00100000:0.0:1789760767.654569:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507310976 00000100:00100000:0.0:1789760767.654634:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748856400607 time 748923775227 state 0x0 ref 1 00000100:00100000:0.0:1789760767.654650:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748924035643 time 748923775227 state 0x1 ref 1 00000100:00100000:0.0:1789760767.654682:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748924035643 time 748969985417 state 0x0 ref 1 00000100:00100000:0.0:1789760767.654733:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.654783:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21663744, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.654793:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.654811:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760767.654816:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.654823:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.654839:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.691633:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00002000:00010000:1.0:1789760767.691764:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21663744 -> 21667840 00010000:00010000:1.0:1789760767.691783:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691798:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691807:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691814:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.691818:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691827:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691848:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.691862:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8cda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.691950:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507310976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37253us (37503us total) trans 4294972609 rc 0/0 00000100:00100000:1.0:1789760767.691959:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5319 00000100:00100000:1.0:1789760767.691968:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748924035643 time 748969985417 state 0x2 ref 0 00000100:00100000:1.0:1789760767.700895:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.701857:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311104 00000100:00100000:0.0:1789760767.701869:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311104 00000100:00100000:0.0:1789760767.702003:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748924035643 time 748969985417 state 0x0 ref 1 00000100:00100000:0.0:1789760767.702018:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748970245833 time 748969985417 state 0x1 ref 1 00000100:00100000:0.0:1789760767.702099:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748970245833 time 749017401776 state 0x0 ref 1 00000100:00100000:0.0:1789760767.702113:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.702165:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21667840, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.702173:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.702188:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760767.702192:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.702198:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.702212:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.751019:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:1.0:1789760767.751160:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:1.0:1789760767.751175:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751188:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751197:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751204:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.751209:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751218:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751242:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.751249:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8ce1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.751370:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49254us (50479us total) trans 4294972610 rc 0/0 00000100:00100000:1.0:1789760767.751380:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5320 00000100:00100000:1.0:1789760767.751390:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748970245833 time 749017401776 state 0x2 ref 0 00000100:00100000:1.0:1789760767.758952:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.759124:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311232 00000100:00100000:0.0:1789760767.759133:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311232 00000100:00100000:0.0:1789760767.759187:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 748970245833 time 749017401776 state 0x0 ref 1 00000100:00100000:0.0:1789760767.759205:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749017662192 time 749017401776 state 0x1 ref 1 00000100:00100000:0.0:1789760767.759263:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749017662192 time 749074565780 state 0x0 ref 1 00000100:00100000:0.0:1789760767.759276:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507311232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.759323:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21671936, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.759333:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.759350:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760767.759354:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.759361:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.759376:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.797511:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:1.0:1789760767.797550:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797564:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797573:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797580:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.797584:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797609:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797637:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.797653:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8ce8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.797786:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507311232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38503us (38833us total) trans 4294972611 rc 0/0 00000100:00100000:1.0:1789760767.797795:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5321 00000100:00100000:1.0:1789760767.797804:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749017662192 time 749074565780 state 0x2 ref 0 00000001:00080000:3.0:1789760767.798457:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00000100:00100000:3.0:1789760767.808547:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760767.808663:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311360 00000100:00100000:1.0:1789760767.808673:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311360 00000100:00100000:1.0:1789760767.808763:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749017662192 time 749074565780 state 0x0 ref 1 00000100:00100000:1.0:1789760767.808779:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749074826196 time 749074565780 state 0x1 ref 1 00000100:00100000:1.0:1789760767.808816:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749074826196 time 749124109250 state 0x0 ref 1 00000100:00100000:1.0:1789760767.808830:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507311360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760767.808879:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21676032, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:1.0:1789760767.808889:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.808905:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760767.808910:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.808917:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.808930:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.857134:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00002000:00010000:1.0:1789760767.857248:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:1.0:1789760767.857264:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857277:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857287:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857294:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.857300:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857310:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857329:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.857338:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8cef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.857467:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507311360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48634us (48924us total) trans 4294972612 rc 0/0 00000100:00100000:1.0:1789760767.857479:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5322 00000100:00100000:1.0:1789760767.857488:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749074826196 time 749124109250 state 0x2 ref 0 00000100:00100000:1.0:1789760767.860944:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.861024:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311488 00000100:00100000:0.0:1789760767.861094:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311488 00000100:00100000:0.0:1789760767.861153:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749074826196 time 749124109250 state 0x0 ref 1 00000100:00100000:0.0:1789760767.861169:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749124369666 time 749124109250 state 0x1 ref 1 00000100:00100000:0.0:1789760767.861198:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749124369666 time 749176501927 state 0x0 ref 1 00000100:00100000:0.0:1789760767.861214:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507311488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.861260:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21680128, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.861269:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.861285:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760767.861289:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.861295:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.861310:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.891277:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00002000:00010000:2.0:1789760767.891374:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:2.0:1789760767.891390:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891404:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891413:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891426:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760767.891431:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891440:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891460:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.891468:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8cf6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.891562:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507311488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30343us (30620us total) trans 4294972613 rc 0/0 00000100:00100000:2.0:1789760767.891572:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5323 00000100:00100000:2.0:1789760767.891582:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749124369666 time 749176501927 state 0x2 ref 0 00010000:00010000:3.0:1789760767.902402:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760767.902839:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760767.902912:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311616 00000100:00100000:0.0:1789760767.902921:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311616 00000100:00100000:0.0:1789760767.902973:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749124369666 time 749176501927 state 0x0 ref 1 00000100:00100000:0.0:1789760767.902989:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749176762343 time 749176501927 state 0x1 ref 1 00000100:00100000:0.0:1789760767.903018:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749176762343 time 749218321187 state 0x0 ref 1 00000100:00100000:0.0:1789760767.903032:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760767.903185:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21684224, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760767.903195:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.903211:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760767.903216:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.903223:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.903236:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.942408:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972614 is committed 00002000:00010000:1.0:1789760767.942504:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:1.0:1789760767.942518:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942531:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942541:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942548:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760767.942553:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942562:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942582:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.942599:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8cfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.942733:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39665us (39861us total) trans 4294972614 rc 0/0 00000100:00100000:1.0:1789760767.942743:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5324 00000100:00100000:1.0:1789760767.942753:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749176762343 time 749218321187 state 0x2 ref 0 00000100:00100000:0.0:1789760767.946572:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760767.946666:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311744 00000100:00100000:1.0:1789760767.946675:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311744 00000100:00100000:1.0:1789760767.946907:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749176762343 time 749218321187 state 0x0 ref 1 00000100:00100000:1.0:1789760767.946923:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749218581603 time 749218321187 state 0x1 ref 1 00000100:00100000:1.0:1789760767.946945:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749218581603 time 749262248251 state 0x0 ref 1 00000100:00100000:1.0:1789760767.946956:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760767.947004:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21688320, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:1.0:1789760767.947013:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.947027:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760767.947032:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.947051:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.947065:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.978131:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00002000:00010000:2.0:1789760767.978482:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:2.0:1789760767.978496:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978509:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978522:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978529:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760767.978534:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978542:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978560:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.978567:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8d04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.978675:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31715us (32108us total) trans 4294972615 rc 0/0 00000100:00100000:2.0:1789760767.978685:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5325 00000100:00100000:2.0:1789760767.978745:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749218581603 time 749262248251 state 0x2 ref 0 00000100:00100000:0.0:1789760767.986786:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760767.986977:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507311872 00000100:00100000:2.0:1789760767.986988:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507311872 00000100:00100000:2.0:1789760767.987052:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749218581603 time 749262248251 state 0x0 ref 1 00000100:00100000:2.0:1789760767.987067:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749262508667 time 749262248251 state 0x1 ref 1 00000100:00100000:2.0:1789760767.987100:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749262508667 time 749302402582 state 0x0 ref 1 00000100:00100000:2.0:1789760767.987113:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760767.987159:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21692416, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:2.0:1789760767.987168:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.987186:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760767.987191:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.987198:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.987213:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.026330:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00002000:00010000:2.0:1789760768.026412:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21692416 -> 21696512 00010000:00010000:2.0:1789760768.026496:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026510:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026519:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026526:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760768.026532:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026542:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026563:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.026571:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.027143:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507311872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40026us (40361us total) trans 4294972616 rc 0/0 00000100:00100000:2.0:1789760768.027153:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5326 00000100:00100000:2.0:1789760768.027162:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749262508667 time 749302402582 state 0x2 ref 0 00000100:00100000:1.0:1789760768.032329:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760768.032492:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312000 00000100:00100000:0.0:1789760768.032502:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312000 00000100:00100000:0.0:1789760768.032557:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749262508667 time 749302402582 state 0x0 ref 1 00000100:00100000:0.0:1789760768.032572:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749302662998 time 749302402582 state 0x1 ref 1 00000100:00100000:0.0:1789760768.032610:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749302662998 time 749347913322 state 0x0 ref 1 00000100:00100000:0.0:1789760768.032621:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507312000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760768.032673:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21696512, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xa807 00010000:00010000:0.0:1789760768.032681:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.032809:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760768.032815:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.032822:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.032838:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.082355:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00002000:00010000:0.0:1789760768.082527:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21696512 -> 21700608 00002000:00010000:0.0:1789760768.082532:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:0.0:1789760768.082535:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:0.0:1789760768.082547:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082560:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082569:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082577:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760768.082582:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082601:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082620:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.082719:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8d12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.082844:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507312000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50220us (50522us total) trans 4294972617 rc 0/0 00000100:00100000:0.0:1789760768.082855:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5327 00000100:00100000:0.0:1789760768.082865:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749302662998 time 749347913322 state 0x2 ref 0 00000100:00100000:0.0:1789760768.086839:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760768.086938:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312128 00000100:00100000:2.0:1789760768.086947:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312128 00000100:00100000:2.0:1789760768.087000:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749302662998 time 749347913322 state 0x0 ref 1 00000100:00100000:2.0:1789760768.087015:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749348173738 time 749347913322 state 0x1 ref 1 00000100:00100000:2.0:1789760768.087145:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749348173738 time 749402345115 state 0x0 ref 1 00000100:00100000:2.0:1789760768.087160:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507312128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.087210:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21700608, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.087219:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.087235:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.087239:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.087245:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.087260:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.120245:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:2.0:1789760768.120264:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120278:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120287:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120294:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760768.120299:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120308:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120332:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.120339:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8d19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.120507:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507312128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33344us (33672us total) trans 4294972618 rc 0/0 00000100:00100000:2.0:1789760768.120519:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5328 00000100:00100000:2.0:1789760768.120530:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749348173738 time 749402345115 state 0x2 ref 0 00000001:00080000:1.0:1789760768.121005:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00000100:00100000:1.0:1789760768.126332:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760768.126412:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312256 00000100:00100000:2.0:1789760768.126432:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312256 00000100:00100000:2.0:1789760768.126491:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749348173738 time 749402345115 state 0x0 ref 1 00000100:00100000:2.0:1789760768.126507:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749402605531 time 749402345115 state 0x1 ref 1 00000100:00100000:2.0:1789760768.126537:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749402605531 time 749441840576 state 0x0 ref 1 00000100:00100000:2.0:1789760768.126549:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.126608:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21704704, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.126617:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.126642:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.126655:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.126662:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.126677:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.161390:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00002000:00010000:2.0:1789760768.165182:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21704704 -> 21708800 00010000:00010000:2.0:1789760768.165202:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165214:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165223:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165229:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760768.165233:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165242:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165264:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.165273:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.165377:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38823us (39047us total) trans 4294972619 rc 0/0 00000100:00100000:2.0:1789760768.165389:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5329 00000100:00100000:2.0:1789760768.165400:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749402605531 time 749441840576 state 0x2 ref 0 00000100:00100000:2.0:1789760768.175916:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760768.176012:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312384 00000100:00100000:0.0:1789760768.176022:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312384 00000100:00100000:0.0:1789760768.176076:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749402605531 time 749441840576 state 0x0 ref 1 00000100:00100000:0.0:1789760768.176092:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749442100992 time 749441840576 state 0x1 ref 1 00000100:00100000:0.0:1789760768.176124:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749442100992 time 749491427198 state 0x0 ref 1 00000100:00100000:0.0:1789760768.176137:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760768.176196:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21708800, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:0.0:1789760768.176206:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.176220:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760768.176225:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.176232:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.176247:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.217273:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972620 is committed 00002000:00010000:3.0:1789760768.217531:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:3.0:1789760768.217547:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217561:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217571:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217580:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.217585:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217611:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217640:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.217650:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8d27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.217781:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41641us (41872us total) trans 4294972620 rc 0/0 00000100:00100000:3.0:1789760768.217791:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5330 00000100:00100000:3.0:1789760768.217799:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749442100992 time 749491427198 state 0x2 ref 0 00000100:00100000:0.0:1789760768.223789:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760768.223937:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312512 00000100:00100000:2.0:1789760768.223947:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312512 00000100:00100000:2.0:1789760768.223998:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749442100992 time 749491427198 state 0x0 ref 1 00000100:00100000:2.0:1789760768.224013:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749491687614 time 749491427198 state 0x1 ref 1 00000100:00100000:2.0:1789760768.224091:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749491687614 time 749539393018 state 0x0 ref 1 00000100:00100000:2.0:1789760768.224105:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507312512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.224151:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21712896, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.224160:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.224176:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.224181:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.224188:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.224203:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.274533:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21712896 -> 21716992 00010000:00010000:1.0:1789760768.274553:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274566:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274575:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274582:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760768.274597:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274613:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274642:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.274650:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.274775:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507312512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50667us (50991us total) trans 4294972621 rc 0/0 00000100:00100000:1.0:1789760768.274784:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5331 00000100:00100000:1.0:1789760768.274793:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749491687614 time 749539393018 state 0x2 ref 0 00000001:00080000:0.0:1789760768.274887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00000100:00100000:0.0:1789760768.290577:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760768.290725:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312640 00000100:00100000:2.0:1789760768.290738:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312640 00000100:00100000:2.0:1789760768.290795:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749491687614 time 749539393018 state 0x0 ref 1 00000100:00100000:2.0:1789760768.290809:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749539653434 time 749539393018 state 0x1 ref 1 00000100:00100000:2.0:1789760768.290838:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749539653434 time 749606141862 state 0x0 ref 1 00000100:00100000:2.0:1789760768.290851:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507312640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.290901:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21716992, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.290910:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.290927:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.290931:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.290938:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.290951:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.333065:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:3.0:1789760768.333086:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333099:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333108:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333116:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.333120:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333135:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333154:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333161:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8d35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.333269:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507312640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42415us (42697us total) trans 4294972622 rc 0/0 00000100:00100000:3.0:1789760768.333280:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5332 00000100:00100000:3.0:1789760768.333289:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749539653434 time 749606141862 state 0x2 ref 0 00000001:00080000:1.0:1789760768.333877:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00000100:00100000:2.0:1789760768.338533:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.338625:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312768 00000100:00100000:3.0:1789760768.338635:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312768 00000100:00100000:3.0:1789760768.338690:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749539653434 time 749606141862 state 0x0 ref 1 00000100:00100000:3.0:1789760768.338816:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749606402278 time 749606141862 state 0x1 ref 1 00000100:00100000:2.0:1789760768.338940:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749606402278 time 749654240672 state 0x0 ref 1 00000100:00100000:2.0:1789760768.338961:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.339004:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21721088, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.339012:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.339028:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.339032:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.339038:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.339162:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.383450:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21721088 -> 21725184 00010000:00010000:3.0:1789760768.383471:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383484:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383500:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383508:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.383513:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383522:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383541:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383549:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8d3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.383670:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507312768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44708us (45138us total) trans 4294972623 rc 0/0 00000100:00100000:3.0:1789760768.383679:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5333 00000100:00100000:3.0:1789760768.383689:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749606402278 time 749654240672 state 0x2 ref 0 00000001:00080000:1.0:1789760768.384822:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00000100:00100000:0.0:1789760768.388367:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760768.388614:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507312896 00000100:00100000:1.0:1789760768.388624:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507312896 00000100:00100000:1.0:1789760768.388676:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749606402278 time 749654240672 state 0x0 ref 1 00000100:00100000:1.0:1789760768.388708:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749654501088 time 749654240672 state 0x1 ref 1 00000100:00100000:1.0:1789760768.388731:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749654501088 time 749704034655 state 0x0 ref 1 00000100:00100000:1.0:1789760768.388743:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760768.388791:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21725184, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:1.0:1789760768.388800:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.388815:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760768.388819:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.388825:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.388838:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.435904:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972624 is committed 00002000:00010000:3.0:1789760768.436163:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:3.0:1789760768.436180:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436200:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436209:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436216:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.436220:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436229:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436248:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.436255:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b8d43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.436348:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507312896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47602us (47986us total) trans 4294972624 rc 0/0 00000100:00100000:3.0:1789760768.436359:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5334 00000100:00100000:3.0:1789760768.436368:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749654501088 time 749704034655 state 0x2 ref 0 00000100:00100000:0.0:1789760768.439895:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.439975:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313024 00000100:00100000:3.0:1789760768.439985:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313024 00000100:00100000:3.0:1789760768.440049:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749654501088 time 749704034655 state 0x0 ref 1 00000100:00100000:3.0:1789760768.440066:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749704295071 time 749704034655 state 0x1 ref 1 00000100:00100000:3.0:1789760768.440096:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749704295071 time 749755399588 state 0x0 ref 1 00000100:00100000:3.0:1789760768.440121:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507313024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.440173:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21729280, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.440182:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.440197:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760768.440202:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.440208:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.440222:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.477140:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21729280 -> 21733376 00010000:00010000:0.0:1789760768.477161:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477184:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477194:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477200:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760768.477205:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477213:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477240:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.477248:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.477340:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507313024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37219us (37448us total) trans 4294972625 rc 0/0 00000100:00100000:0.0:1789760768.477350:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5335 00000100:00100000:0.0:1789760768.477359:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749704295071 time 749755399588 state 0x2 ref 0 00000001:00080000:0.0:1789760768.477399:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00000100:00100000:2.0:1789760768.487874:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.487959:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313152 00000100:00100000:3.0:1789760768.487970:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313152 00000100:00100000:3.0:1789760768.488019:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749704295071 time 749755399588 state 0x0 ref 1 00000100:00100000:3.0:1789760768.488034:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749755660004 time 749755399588 state 0x1 ref 1 00000100:00100000:3.0:1789760768.488180:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749755660004 time 749803481977 state 0x0 ref 1 00000100:00100000:3.0:1789760768.488198:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.488244:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21733376, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.488254:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.488270:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760768.488407:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.488416:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.488431:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.516490:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00002000:00010000:3.0:1789760768.516600:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21733376 -> 21737472 00010000:00010000:3.0:1789760768.516616:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516639:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516649:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516656:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.516661:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516670:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516690:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.516719:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.516815:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28615us (28945us total) trans 4294972626 rc 0/0 00000100:00100000:3.0:1789760768.516825:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5336 00000100:00100000:3.0:1789760768.516835:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749755660004 time 749803481977 state 0x2 ref 0 00000100:00100000:1.0:1789760768.524351:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.524450:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313280 00000100:00100000:3.0:1789760768.524460:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313280 00000100:00100000:3.0:1789760768.524512:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749755660004 time 749803481977 state 0x0 ref 1 00000100:00100000:3.0:1789760768.524529:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749803742393 time 749803481977 state 0x1 ref 1 00000100:00100000:3.0:1789760768.524599:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749803742393 time 749839902381 state 0x0 ref 1 00000100:00100000:3.0:1789760768.524611:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.524666:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21737472, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.524676:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.524707:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760768.524713:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.524719:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.524734:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.555536:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00002000:00010000:0.0:1789760768.555671:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:0.0:1789760768.555688:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555727:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555738:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555750:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760768.555755:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555769:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555789:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.555797:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b8d58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.555891:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31276us (31544us total) trans 4294972627 rc 0/0 00000100:00100000:0.0:1789760768.555901:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5337 00000100:00100000:0.0:1789760768.555911:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749803742393 time 749839902381 state 0x2 ref 0 00000100:00100000:0.0:1789760768.559977:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.560059:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313408 00000100:00100000:3.0:1789760768.560069:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313408 00000100:00100000:3.0:1789760768.560133:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749803742393 time 749839902381 state 0x0 ref 1 00000100:00100000:3.0:1789760768.560148:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749840162797 time 749839902381 state 0x1 ref 1 00000100:00100000:3.0:1789760768.560176:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749840162797 time 749875479168 state 0x0 ref 1 00000100:00100000:3.0:1789760768.560189:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.560234:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21741568, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.560243:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.560257:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760768.560262:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.560269:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.560300:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.587631:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00002000:00010000:1.0:1789760768.587788:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21741568 -> 21745664 00010000:00010000:1.0:1789760768.587803:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587816:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587826:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587834:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760768.587839:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587853:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587874:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.587881:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8d5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.587969:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27778us (27997us total) trans 4294972628 rc 0/0 00000100:00100000:1.0:1789760768.587979:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5338 00000100:00100000:1.0:1789760768.587989:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749840162797 time 749875479168 state 0x2 ref 0 00000100:00100000:2.0:1789760768.592260:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.592368:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313536 00000100:00100000:3.0:1789760768.592378:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313536 00000100:00100000:3.0:1789760768.592429:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749840162797 time 749875479168 state 0x0 ref 1 00000100:00100000:3.0:1789760768.592444:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749875739584 time 749875479168 state 0x1 ref 1 00000100:00100000:3.0:1789760768.592503:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749875739584 time 749907805582 state 0x0 ref 1 00000100:00100000:3.0:1789760768.592516:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507313536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.592563:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21745664, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.592571:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.592602:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760768.592608:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.592614:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.592636:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.625865:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21745664 -> 21749760 00010000:00010000:2.0:1789760768.625882:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625896:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625905:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625912:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760768.625917:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625927:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625947:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.625954:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b8d66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.626202:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507313536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33683us (33946us total) trans 4294972629 rc 0/0 00000100:00100000:2.0:1789760768.626213:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5339 00000100:00100000:2.0:1789760768.626223:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749875739584 time 749907805582 state 0x2 ref 0 00000001:00080000:2.0:1789760768.626270:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972629 is committed 00000100:00100000:0.0:1789760768.630238:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.632053:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313664 00000100:00100000:3.0:1789760768.632067:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313664 00000100:00100000:2.0:1789760768.632257:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749875739584 time 749907805582 state 0x0 ref 1 00000100:00100000:2.0:1789760768.632274:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749908065998 time 749907805582 state 0x1 ref 1 00000100:00100000:2.0:1789760768.632440:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749908065998 time 749947741604 state 0x0 ref 1 00000100:00100000:2.0:1789760768.632455:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.632509:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21749760, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.632521:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.632538:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760768.632544:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.632550:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.632564:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.669737:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00002000:00010000:3.0:1789760768.669904:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:3.0:1789760768.669919:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669932:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669941:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669948:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.669953:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669962:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669980:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.669988:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b8d6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.670100:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507313664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37644us (39868us total) trans 4294972630 rc 0/0 00000100:00100000:3.0:1789760768.670110:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5340 00000100:00100000:3.0:1789760768.670119:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749908065998 time 749947741604 state 0x2 ref 0 00000100:00100000:0.0:1789760768.675415:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760768.675498:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313792 00000100:00100000:1.0:1789760768.675508:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313792 00000100:00100000:1.0:1789760768.675559:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749908065998 time 749947741604 state 0x0 ref 1 00000100:00100000:1.0:1789760768.675574:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749948002020 time 749947741604 state 0x1 ref 1 00000100:00100000:1.0:1789760768.675644:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749948002020 time 749990947472 state 0x0 ref 1 00000100:00100000:1.0:1789760768.675656:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507313792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760768.675728:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21753856, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:1.0:1789760768.675738:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.675753:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760768.675758:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.675764:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.675777:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.725976:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.726019:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.726055:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.726077:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.726111:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760768.726121:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314048 00000100:00100000:2.0:1789760768.726130:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314048 00000100:00100000:3.0:1789760768.726131:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507313920 00000100:00100000:3.0:1789760768.726133:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507313920 00000100:00100000:2.0:1789760768.726171:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314304 00000100:00100000:2.0:1789760768.726174:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314304 00000100:00100000:2.0:1789760768.726194:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314176 00000100:00100000:2.0:1789760768.726198:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314176 00000100:00100000:2.0:1789760768.726212:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314432 00000100:00100000:2.0:1789760768.726215:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314432 00000100:00100000:2.0:1789760768.726230:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 938 00000100:00100000:2.0:1789760768.726239:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507314048:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.726310:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507314048:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 72us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.726318:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 938 00000100:00100000:2.0:1789760768.726335:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 939 00000100:00100000:2.0:1789760768.726342:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507314304:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.726368:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507314304:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 27us (348us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.726375:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 939 00000100:00100000:2.0:1789760768.726386:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 940 00000100:00100000:2.0:1789760768.726392:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507314176:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.726413:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507314176:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 22us (335us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.726418:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 940 00000100:00100000:2.0:1789760768.726428:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 941 00000100:00100000:2.0:1789760768.726434:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507314432:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.726456:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234675500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507314432:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.726462:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 941 00000100:00100000:3.0:1789760768.726648:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 478 00000100:00100000:3.0:1789760768.726656:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507313920:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.726867:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507313920:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 211us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.726874:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 478 00000100:00080000:3.0:1789760768.726911:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760768.726915:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:3.0:1789760768.731569:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21753856 -> 21757952 00000001:00080000:0.0:1789760768.731637:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00010000:00010000:3.0:1789760768.731885:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731899:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731908:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731916:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.731920:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731930:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731953:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.731961:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b8d74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.732061:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507313792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56402us (56651us total) trans 4294972631 rc 0/0 00000100:00100000:3.0:1789760768.732072:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5341 00000100:00100000:3.0:1789760768.732081:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749948002020 time 749990947472 state 0x2 ref 0 00000100:00100000:2.0:1789760768.735635:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.735826:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314560 00000100:00100000:3.0:1789760768.735834:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314560 00000100:00100000:3.0:1789760768.735879:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749948002020 time 749990947472 state 0x0 ref 1 00000100:00100000:3.0:1789760768.735890:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749991207888 time 749990947472 state 0x1 ref 1 00000100:00100000:3.0:1789760768.735936:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749991207888 time 750051238594 state 0x0 ref 1 00000100:00100000:3.0:1789760768.735948:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507314560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.736005:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21757952, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.736012:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.736025:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760768.736028:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.736181:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.736195:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.782867:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:3.0:1789760768.783080:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21757952 -> 21762048 00010000:00010000:3.0:1789760768.783165:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783178:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783188:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783195:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.783199:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783209:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783231:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.783239:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.783335:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507314560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47383us (47705us total) trans 4294972632 rc 0/0 00000100:00100000:3.0:1789760768.783345:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5342 00000100:00100000:3.0:1789760768.783355:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749991207888 time 750051238594 state 0x2 ref 0 00000100:00100000:3.0:1789760768.786752:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760768.787622:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314688 00000100:00100000:0.0:1789760768.787633:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314688 00000100:00100000:0.0:1789760768.787689:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 749991207888 time 750051238594 state 0x0 ref 1 00000100:00100000:0.0:1789760768.787732:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750051499010 time 750051238594 state 0x1 ref 1 00000100:00100000:0.0:1789760768.787760:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750051499010 time 750103062802 state 0x0 ref 1 00000100:00100000:0.0:1789760768.787773:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507314688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760768.787864:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21762048, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:2.0:1789760768.787874:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.787889:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760768.787893:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.787900:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.787914:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.824896:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00002000:00010000:0.0:1789760768.825077:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:0.0:1789760768.825161:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825176:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825186:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825193:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760768.825199:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825208:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825229:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825236:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b8d82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.825368:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507314688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37591us (38618us total) trans 4294972633 rc 0/0 00000100:00100000:0.0:1789760768.825378:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5343 00000100:00100000:0.0:1789760768.825443:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750051499010 time 750103062802 state 0x2 ref 0 00000100:00100000:1.0:1789760768.829951:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.830830:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314816 00000100:00100000:3.0:1789760768.830848:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314816 00000100:00100000:3.0:1789760768.831163:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750051499010 time 750103062802 state 0x0 ref 1 00000100:00100000:3.0:1789760768.831183:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750103323218 time 750103062802 state 0x1 ref 1 00000100:00100000:3.0:1789760768.831221:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750103323218 time 750146523500 state 0x0 ref 1 00000100:00100000:3.0:1789760768.831239:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507314816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.831311:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21766144, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.831382:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.831405:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760768.831411:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.831419:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.831437:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.859227:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21766144 -> 21770240 00010000:00010000:1.0:1789760768.859247:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859260:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859271:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859278:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760768.859283:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859293:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859313:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.859322:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8d89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.859627:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00000100:00100000:1.0:1789760768.859806:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507314816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28568us (30216us total) trans 4294972634 rc 0/0 00000100:00100000:1.0:1789760768.859818:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5344 00000100:00100000:1.0:1789760768.859828:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750103323218 time 750146523500 state 0x2 ref 0 00000100:00100000:2.0:1789760768.863825:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.863890:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507314944 00000100:00100000:3.0:1789760768.863899:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507314944 00000100:00100000:3.0:1789760768.863956:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750103323218 time 750146523500 state 0x0 ref 1 00000100:00100000:3.0:1789760768.863970:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750146783916 time 750146523500 state 0x1 ref 1 00000100:00100000:3.0:1789760768.864005:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750146783916 time 750179309405 state 0x0 ref 1 00000100:00100000:3.0:1789760768.864016:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507314944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.864159:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21770240, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.864169:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.864185:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760768.864190:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.864196:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.864210:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.913152:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00002000:00010000:3.0:1789760768.914611:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:3.0:1789760768.914716:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914728:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914736:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914743:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.914748:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914756:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914776:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.914784:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.914898:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507314944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50879us (51079us total) trans 4294972635 rc 0/0 00000100:00100000:3.0:1789760768.914908:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5345 00000100:00100000:3.0:1789760768.914918:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750146783916 time 750179309405 state 0x2 ref 0 00000100:00100000:3.0:1789760768.919042:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760768.919842:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315072 00000100:00100000:0.0:1789760768.919854:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315072 00000100:00100000:0.0:1789760768.919919:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750146783916 time 750179309405 state 0x0 ref 1 00000100:00100000:0.0:1789760768.919934:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750179569821 time 750179309405 state 0x1 ref 1 00000100:00100000:0.0:1789760768.919967:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750179569821 time 750235270606 state 0x0 ref 1 00000100:00100000:0.0:1789760768.919979:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760768.920025:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21774336, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:0.0:1789760768.920044:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.920060:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760768.920064:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.920070:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.920084:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.924120:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760768.957135:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00002000:00010000:2.0:1789760768.957313:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:2.0:1789760768.957330:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957343:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957353:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957360:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760768.957365:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957374:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957392:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.957399:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b8d97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.957493:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37511us (38469us total) trans 4294972636 rc 0/0 00000100:00100000:2.0:1789760768.957504:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5346 00000100:00100000:2.0:1789760768.957514:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750179569821 time 750235270606 state 0x2 ref 0 00000100:00100000:1.0:1789760768.961057:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760768.961331:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315200 00000100:00100000:3.0:1789760768.961342:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315200 00000100:00100000:3.0:1789760768.961394:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750179569821 time 750235270606 state 0x0 ref 1 00000100:00100000:3.0:1789760768.961409:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750235531022 time 750235270606 state 0x1 ref 1 00000100:00100000:3.0:1789760768.961778:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750235531022 time 750277079854 state 0x0 ref 1 00000100:00100000:3.0:1789760768.961793:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760768.961838:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21778432, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760768.961847:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.961862:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760768.961867:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.961873:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.961888:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.994549:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00002000:00010000:3.0:1789760768.995231:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21778432 -> 21782528 00010000:00010000:3.0:1789760768.995259:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995276:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995285:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995292:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760768.995296:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995305:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995327:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.995334:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8d9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.995431:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33636us (34380us total) trans 4294972637 rc 0/0 00000100:00100000:3.0:1789760768.995442:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5347 00000100:00100000:3.0:1789760768.995451:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750235531022 time 750277079854 state 0x2 ref 0 00000100:00100000:0.0:1789760769.000644:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760769.000718:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315328 00000100:00100000:3.0:1789760769.000727:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315328 00000100:00100000:3.0:1789760769.000806:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750235531022 time 750277079854 state 0x0 ref 1 00000100:00100000:3.0:1789760769.000821:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750277340270 time 750277079854 state 0x1 ref 1 00000100:00100000:3.0:1789760769.000851:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750277340270 time 750316154047 state 0x0 ref 1 00000100:00100000:3.0:1789760769.000863:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760769.000908:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21782528, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xa807 00010000:00010000:3.0:1789760769.000919:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.000933:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760769.000937:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.000945:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.000959:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760769.001410:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760769.040780:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21782528 -> 21786624 00002000:00010000:1.0:1789760769.040789:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:1.0:1789760769.040792:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:1.0:1789760769.040806:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040818:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040827:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040838:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760769.040843:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040851:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040884:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.040892:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8da5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.040986:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507315328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40120us (40347us total) trans 4294972638 rc 0/0 00000100:00100000:1.0:1789760769.040996:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5348 00000100:00100000:1.0:1789760769.041006:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750277340270 time 750316154047 state 0x2 ref 0 00000001:00080000:2.0:1789760769.041271:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00000100:00100000:2.0:1789760769.047013:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.047167:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315456 00000100:00100000:0.0:1789760769.047177:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315456 00000100:00100000:0.0:1789760769.047230:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750277340270 time 750316154047 state 0x0 ref 1 00000100:00100000:0.0:1789760769.047245:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750316414463 time 750316154047 state 0x1 ref 1 00000100:00100000:0.0:1789760769.047275:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750316414463 time 750362578147 state 0x0 ref 1 00000100:00100000:0.0:1789760769.047288:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507315456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.047335:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21786624, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.047344:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.047360:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.047365:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.047371:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.047385:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.073914:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:0.0:1789760769.073934:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.073946:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.073956:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.073964:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.073968:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.073978:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.073998:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.074006:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.074101:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507315456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26811us (27092us total) trans 4294972639 rc 0/0 00000100:00100000:0.0:1789760769.074112:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5349 00000100:00100000:0.0:1789760769.074123:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750316414463 time 750362578147 state 0x2 ref 0 00000001:00080000:1.0:1789760769.074201:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00000100:00100000:0.0:1789760769.082867:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760769.083140:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315584 00000100:00100000:3.0:1789760769.083151:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315584 00000100:00100000:3.0:1789760769.083203:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750316414463 time 750362578147 state 0x0 ref 1 00000100:00100000:3.0:1789760769.083218:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750362838563 time 750362578147 state 0x1 ref 1 00000100:00100000:3.0:1789760769.083287:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750362838563 time 750398589709 state 0x0 ref 1 00000100:00100000:3.0:1789760769.083306:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760769.083358:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21790720, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:3.0:1789760769.083366:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.083383:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760769.083388:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.083394:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.083409:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.112673:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00002000:00010000:2.0:1789760769.113075:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21790720 -> 21794816 00010000:00010000:2.0:1789760769.113090:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113102:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113110:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113116:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760769.113121:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113128:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113146:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113153:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8db3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.113276:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507315584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29969us (30414us total) trans 4294972640 rc 0/0 00000100:00100000:2.0:1789760769.113287:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5350 00000100:00100000:2.0:1789760769.113297:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750362838563 time 750398589709 state 0x2 ref 0 00000100:00100000:3.0:1789760769.117419:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760769.117798:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315712 00000100:00100000:1.0:1789760769.117809:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315712 00000100:00100000:1.0:1789760769.117859:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750362838563 time 750398589709 state 0x0 ref 1 00000100:00100000:1.0:1789760769.117874:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750398850125 time 750398589709 state 0x1 ref 1 00000100:00100000:1.0:1789760769.117908:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750398850125 time 750433211428 state 0x0 ref 1 00000100:00100000:1.0:1789760769.117921:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507315712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.117966:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21794816, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.117975:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.117989:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760769.117994:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.118001:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.118014:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.156933:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21794816 -> 21798912 00010000:00010000:3.0:1789760769.156956:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.156970:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.156980:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.156987:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760769.156992:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.157001:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.157023:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.157043:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b8dba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.157139:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507315712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39215us (39724us total) trans 4294972641 rc 0/0 00000100:00100000:3.0:1789760769.157149:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5351 00000100:00100000:3.0:1789760769.157159:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750398850125 time 750433211428 state 0x2 ref 0 00000100:00100000:3.0:1789760769.160575:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760769.160738:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315840 00000100:00100000:1.0:1789760769.160748:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315840 00000100:00100000:1.0:1789760769.160800:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750398850125 time 750433211428 state 0x0 ref 1 00000100:00100000:1.0:1789760769.160815:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750433471844 time 750433211428 state 0x1 ref 1 00000100:00100000:1.0:1789760769.160836:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750433471844 time 750476139634 state 0x0 ref 1 00000100:00100000:1.0:1789760769.160848:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507315840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.160895:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21798912, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.160905:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.160920:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760769.160925:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.160931:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.160945:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.161807:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00000001:00080000:3.0:1789760769.189288:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00002000:00010000:0.0:1789760769.189392:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21798912 -> 21803008 00010000:00010000:0.0:1789760769.189407:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189420:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189429:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189436:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.189441:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189450:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189468:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.189476:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8dc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.189757:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507315840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28907us (29187us total) trans 4294972642 rc 0/0 00000100:00100000:0.0:1789760769.189769:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5352 00000100:00100000:0.0:1789760769.189778:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750433471844 time 750476139634 state 0x2 ref 0 00000100:00100000:2.0:1789760769.193596:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760769.193658:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507315968 00000100:00100000:3.0:1789760769.193668:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507315968 00000100:00100000:3.0:1789760769.193739:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750433471844 time 750476139634 state 0x0 ref 1 00000100:00100000:3.0:1789760769.193753:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750476400050 time 750476139634 state 0x1 ref 1 00000100:00100000:3.0:1789760769.193775:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750476400050 time 750509078891 state 0x0 ref 1 00000100:00100000:3.0:1789760769.193786:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507315968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760769.193832:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21803008, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:3.0:1789760769.193841:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193856:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760769.193860:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.193866:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193880:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.226033:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00002000:00010000:3.0:1789760769.226407:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21803008 -> 21807104 00010000:00010000:3.0:1789760769.226424:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226438:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226448:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226455:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760769.226460:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226469:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226562:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.226572:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8dc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.226880:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507315968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33089us (33296us total) trans 4294972643 rc 0/0 00000100:00100000:3.0:1789760769.226891:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5353 00000100:00100000:3.0:1789760769.226900:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750476400050 time 750509078891 state 0x2 ref 0 00000100:00100000:3.0:1789760769.230315:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.230466:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316096 00000100:00100000:0.0:1789760769.230477:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316096 00000100:00100000:0.0:1789760769.230544:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750476400050 time 750509078891 state 0x0 ref 1 00000100:00100000:0.0:1789760769.230560:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750509339307 time 750509078891 state 0x1 ref 1 00000100:00100000:0.0:1789760769.230600:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750509339307 time 750545903195 state 0x0 ref 1 00000100:00100000:0.0:1789760769.230624:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507316096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.230676:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21807104, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.230685:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.230733:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.230737:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.230744:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.230759:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.244953:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760769.244976:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538285440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760769.245116:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.245264:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285440 00000100:00100000:0.0:1789760769.245272:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285440 00000100:00100000:0.0:1789760769.245324:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789760769.245334:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538285440:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760769.245367:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760769.245413:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538285440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.245420:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00080000:0.0:1789760769.245440:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760769.245471:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538285440:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760769.245812:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538285568:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760769.245847:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.245878:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538286080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.245901:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.245927:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538286592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.245951:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.245968:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538287104:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.245986:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.246003:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789760769.246007:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538285696:0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.246126:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538285824:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.246182:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.246253:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285568 00000100:00100000:1.0:1789760769.246259:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285568 00000100:00100000:1.0:1789760769.246298:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:1.0:1789760769.246306:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538285568:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760769.246320:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760769.246388:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.246411:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538286336:0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.246433:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.246451:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538286848:0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.246471:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.246733:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.246764:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538286464:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.246786:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.246805:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538286976:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.246827:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.247012:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285952 00000100:00100000:0.0:1789760769.247017:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285952 00000100:00100000:0.0:1789760769.247063:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1789760769.247070:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538285952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.247114:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538285952:12345-0@lo:400:kworker.0 Request processed in 44us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.247120:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760769.247224:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286080 00000100:00100000:2.0:1789760769.247228:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286080 00000100:00100000:2.0:1789760769.247249:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285824 00000100:00100000:2.0:1789760769.247252:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285824 00000100:00100000:2.0:1789760769.247264:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286336 00000100:00100000:2.0:1789760769.247266:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286336 00000100:00100000:2.0:1789760769.247280:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286848 00000100:00100000:2.0:1789760769.247283:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286848 00000100:00100000:2.0:1789760769.247294:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286464 00000100:00100000:2.0:1789760769.247296:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286464 00000100:00100000:2.0:1789760769.247309:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286976 00000100:00100000:2.0:1789760769.247311:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286976 00000100:00100000:2.0:1789760769.247326:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 942 00000100:00100000:2.0:1789760769.247332:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538286080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247366:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538286080:12345-0@lo:400:kworker.0 Request processed in 34us (1464us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.247373:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 942 00000100:00100000:2.0:1789760769.247387:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 943 00000100:00100000:2.0:1789760769.247393:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538285824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247434:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538285824:12345-0@lo:400:kworker.0 Request processed in 41us (1045us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.247439:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 943 00000100:00100000:2.0:1789760769.247451:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 944 00000100:00100000:2.0:1789760769.247458:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538286336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247486:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538286336:12345-0@lo:400:kworker.0 Request processed in 29us (1052us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.247609:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538285824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.247629:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538286336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247759:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 944 00000100:00100000:2.0:1789760769.247774:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 945 00000100:00100000:2.0:1789760769.247780:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538286848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247813:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108043480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538286848:12345-0@lo:400:kworker.0 Request processed in 33us (1340us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.247819:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 945 00000100:00100000:0.0:1789760769.247907:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538286848:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.247956:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 946 00000100:00100000:2.0:1789760769.247964:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538286464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.248001:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3462:x1876699538286464:12345-0@lo:400:kworker.0 Request processed in 39us (1214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.248007:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 946 00000100:00100000:2.0:1789760769.248018:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 947 00000100:00100000:2.0:1789760769.248097:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538286976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.248131:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3462:x1876699538286976:12345-0@lo:400:kworker.0 Request processed in 107us (1303us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.248137:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 947 00000100:00100000:2.0:1789760769.248166:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538285952:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.248175:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3462:1876699538286464:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.248184:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3462:1876699538286976:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.248320:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538285568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2014us (2470us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.248329:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00080000:1.0:1789760769.248372:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760769.248377:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760769.248380:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760769.248936:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.248992:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538286208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.249153:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538285696 00000100:00100000:1.0:1789760769.249160:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538285696 00000100:00100000:1.0:1789760769.249200:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:1.0:1789760769.249207:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538285696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.249243:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538285696:12345-0@lo:400:kworker.0 Request processed in 36us (625us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.249252:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760769.249508:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.249542:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538286720:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.249568:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760769.249773:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286208 00000100:00100000:1.0:1789760769.249778:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286208 00000100:00100000:1.0:1789760769.249809:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1789760769.249816:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538286208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.249850:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538286208:12345-0@lo:400:kworker.0 Request processed in 34us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.249856:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1789760769.249923:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538285568:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760769.249941:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538286080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.249964:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538285696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.249973:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538286208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.250004:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286592 00000100:00100000:3.0:1789760769.250008:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286592 00000100:00080000:1.0:1789760769.250115:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760769.250752:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287104 00000100:00100000:3.0:1789760769.250758:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287104 00000100:00100000:1.0:1789760769.251169:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538286720 00000100:00100000:1.0:1789760769.251174:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538286720 00000100:00100000:1.0:1789760769.251220:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789760769.251231:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538286592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.251297:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538286592:12345-0@lo:400:kworker.0 Request processed in 70us (5346us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.251304:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789760769.251316:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:1.0:1789760769.251322:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538287104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.251349:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538287104:12345-0@lo:400:kworker.0 Request processed in 27us (5362us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.251354:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:1.0:1789760769.251364:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1789760769.251370:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538286720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.251397:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538286720:12345-0@lo:400:kworker.0 Request processed in 28us (1828us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760769.251403:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00080000:0.0:1789760769.251461:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760769.251689:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538286720:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.252208:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538286592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.252219:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538287104:0@lo:400:kworker.0 00002000:00010000:0.0:1789760769.273398:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21807104 -> 21811200 00010000:00010000:0.0:1789760769.273417:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273430:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273440:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273447:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.273453:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273462:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273482:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.273566:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8dcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.273754:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507316096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43137us (43443us total) trans 4294972644 rc 0/0 00000100:00100000:0.0:1789760769.273765:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5354 00000100:00100000:0.0:1789760769.273774:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750509339307 time 750545903195 state 0x2 ref 0 00000001:00080000:2.0:1789760769.276204:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972644 is committed 00000100:00100000:2.0:1789760769.276786:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.277037:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316224 00000100:00100000:0.0:1789760769.277048:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316224 00000100:00100000:0.0:1789760769.277177:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750509339307 time 750545903195 state 0x0 ref 1 00000100:00100000:0.0:1789760769.277194:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750546163611 time 750545903195 state 0x1 ref 1 00000100:00100000:0.0:1789760769.277267:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750546163611 time 750592569803 state 0x0 ref 1 00000100:00100000:0.0:1789760769.277281:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507316224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.277332:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21811200, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.277342:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277360:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.277366:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.277373:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277388:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.315603:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00002000:00010000:2.0:1789760769.315712:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:2.0:1789760769.315729:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315754:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315765:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315773:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760769.315777:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315787:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315812:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.315819:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8dd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.315919:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507316224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38637us (39134us total) trans 4294972645 rc 0/0 00000100:00100000:2.0:1789760769.316056:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5355 00000100:00100000:2.0:1789760769.316068:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750546163611 time 750592569803 state 0x2 ref 0 00000100:00100000:0.0:1789760769.323150:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760769.323577:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316352 00000100:00100000:3.0:1789760769.323596:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316352 00000100:00100000:3.0:1789760769.323659:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750546163611 time 750592569803 state 0x0 ref 1 00000100:00100000:3.0:1789760769.323675:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750592830219 time 750592569803 state 0x1 ref 1 00000100:00100000:1.0:1789760769.323796:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750592830219 time 750639094571 state 0x0 ref 1 00000100:00100000:1.0:1789760769.323813:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.323864:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21815296, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.323873:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.323888:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760769.323893:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.323900:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.323914:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.384437:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00002000:00010000:1.0:1789760769.384511:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:1.0:1789760769.384526:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384537:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384545:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384553:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760769.384559:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384568:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384597:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.384606:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ddd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.384731:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60882us (61552us total) trans 4294972646 rc 0/0 00000100:00100000:1.0:1789760769.384742:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5356 00000100:00100000:1.0:1789760769.384752:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750592830219 time 750639094571 state 0x2 ref 0 00000100:00100000:3.0:1789760769.392887:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.393179:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316480 00000100:00100000:0.0:1789760769.393189:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316480 00000100:00100000:0.0:1789760769.393256:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750592830219 time 750639094571 state 0x0 ref 1 00000100:00100000:0.0:1789760769.393275:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750639354987 time 750639094571 state 0x1 ref 1 00000100:00100000:0.0:1789760769.393302:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750639354987 time 750708605484 state 0x0 ref 1 00000100:00100000:0.0:1789760769.393314:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507316480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.393358:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21819392, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.393367:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.393383:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.393387:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.393393:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.393405:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.431637:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21819392 -> 21823488 00010000:00010000:0.0:1789760769.431658:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431671:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431680:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431720:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.431726:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431736:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431756:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.431763:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8de4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.431847:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507316480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38529us (38963us total) trans 4294972647 rc 0/0 00000100:00100000:0.0:1789760769.431856:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5357 00000100:00100000:0.0:1789760769.431866:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750639354987 time 750708605484 state 0x2 ref 0 00000001:00080000:1.0:1789760769.433212:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00000100:00100000:2.0:1789760769.435356:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.435490:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316608 00000100:00100000:0.0:1789760769.435500:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316608 00000100:00100000:0.0:1789760769.435553:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750639354987 time 750708605484 state 0x0 ref 1 00000100:00100000:0.0:1789760769.435569:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750708865900 time 750708605484 state 0x1 ref 1 00000100:00100000:0.0:1789760769.435604:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750708865900 time 750750907556 state 0x0 ref 1 00000100:00100000:0.0:1789760769.435620:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.435667:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21823488, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.435676:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.435728:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.435733:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.435741:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.435755:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.478871:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00002000:00010000:0.0:1789760769.479074:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:0.0:1789760769.479090:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479103:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479112:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479120:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.479125:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479135:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479155:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.479163:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8deb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.479328:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43703us (43975us total) trans 4294972648 rc 0/0 00000100:00100000:0.0:1789760769.479339:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5358 00000100:00100000:0.0:1789760769.479350:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750708865900 time 750750907556 state 0x2 ref 0 00000100:00100000:2.0:1789760769.482494:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760769.482797:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316736 00000100:00100000:1.0:1789760769.482807:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316736 00000100:00100000:1.0:1789760769.482859:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750708865900 time 750750907556 state 0x0 ref 1 00000100:00100000:1.0:1789760769.482873:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750751167972 time 750750907556 state 0x1 ref 1 00000100:00100000:1.0:1789760769.482991:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750751167972 time 750798229246 state 0x0 ref 1 00000100:00100000:1.0:1789760769.483006:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507316736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.483050:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21827584, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.483059:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.483073:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760769.483077:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.483083:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.483096:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.551255:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00002000:00010000:1.0:1789760769.551322:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21827584 -> 21831680 00010000:00010000:1.0:1789760769.551339:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551352:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551361:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551368:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760769.551373:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551381:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551401:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.551409:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8df2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.551505:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507316736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 68493us (69010us total) trans 4294972649 rc 0/0 00000100:00100000:1.0:1789760769.551516:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5359 00000100:00100000:1.0:1789760769.551526:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750751167972 time 750798229246 state 0x2 ref 0 00000100:00100000:3.0:1789760769.555945:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.556368:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316864 00000100:00100000:0.0:1789760769.556379:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316864 00000100:00100000:0.0:1789760769.556437:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750751167972 time 750798229246 state 0x0 ref 1 00000100:00100000:0.0:1789760769.556454:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750798489662 time 750798229246 state 0x1 ref 1 00000100:00100000:0.0:1789760769.556495:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750798489662 time 750871789242 state 0x0 ref 1 00000100:00100000:0.0:1789760769.556509:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.556558:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21831680, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.556568:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556593:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.556598:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.556604:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.557587:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.596097:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972650 is committed 00002000:00010000:0.0:1789760769.596189:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21831680 -> 21835776 00010000:00010000:0.0:1789760769.596204:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596216:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596225:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596232:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.596237:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596247:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596270:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.596278:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8df9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.596373:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39862us (40432us total) trans 4294972650 rc 0/0 00000100:00100000:0.0:1789760769.596383:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5360 00000100:00100000:0.0:1789760769.596392:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750798489662 time 750871789242 state 0x2 ref 0 00000100:00100000:3.0:1789760769.602039:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.602103:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507316992 00000100:00100000:0.0:1789760769.602113:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507316992 00000100:00100000:0.0:1789760769.602164:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750798489662 time 750871789242 state 0x0 ref 1 00000100:00100000:0.0:1789760769.602177:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750872049658 time 750871789242 state 0x1 ref 1 00000100:00100000:0.0:1789760769.602207:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750872049658 time 750917511196 state 0x0 ref 1 00000100:00100000:0.0:1789760769.602219:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.602262:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21835776, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.602271:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.602286:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760769.602291:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.602297:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.602311:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.635297:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00002000:00010000:0.0:1789760769.635378:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21835776 -> 21839872 00010000:00010000:0.0:1789760769.635391:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635401:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635407:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635412:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.635415:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635421:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635436:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.635441:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8e00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.635760:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507316992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33537us (33736us total) trans 4294972651 rc 0/0 00000100:00100000:0.0:1789760769.635771:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5361 00000100:00100000:0.0:1789760769.635780:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750872049658 time 750917511196 state 0x2 ref 0 00000100:00100000:2.0:1789760769.643870:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760769.643964:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317120 00000100:00100000:1.0:1789760769.643974:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317120 00000100:00100000:1.0:1789760769.644023:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750872049658 time 750917511196 state 0x0 ref 1 00000100:00100000:1.0:1789760769.644038:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750917771612 time 750917511196 state 0x1 ref 1 00000100:00100000:1.0:1789760769.644161:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750917771612 time 750959463727 state 0x0 ref 1 00000100:00100000:1.0:1789760769.644175:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.644222:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21839872, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.644232:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.644253:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760769.644258:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.644264:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.644276:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.694229:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00002000:00010000:1.0:1789760769.694331:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21839872 -> 21843968 00010000:00010000:1.0:1789760769.694347:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694360:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694371:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694378:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760769.694383:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694392:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694488:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.694498:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.694855:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50676us (50989us total) trans 4294972652 rc 0/0 00000100:00100000:1.0:1789760769.694866:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5362 00000100:00100000:1.0:1789760769.694876:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750917771612 time 750959463727 state 0x2 ref 0 00000100:00100000:3.0:1789760769.699190:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.700143:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317248 00000100:00100000:0.0:1789760769.700154:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317248 00000100:00100000:0.0:1789760769.701830:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750917771612 time 750959463727 state 0x0 ref 1 00000100:00100000:0.0:1789760769.701852:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750959724143 time 750959463727 state 0x1 ref 1 00000100:00100000:1.0:1789760769.701994:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750959724143 time 751017294264 state 0x0 ref 1 00000100:00100000:1.0:1789760769.702020:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507317248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.702153:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21843968, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.702164:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.702181:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760769.702196:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.702204:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.702221:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.741598:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972653 is committed 00002000:00010000:2.0:1789760769.741656:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21843968 -> 21848064 00010000:00010000:2.0:1789760769.741673:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741686:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741723:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741732:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760769.741739:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741749:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741771:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.741779:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.741892:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507317248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39871us (42698us total) trans 4294972653 rc 0/0 00000100:00100000:2.0:1789760769.741904:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5363 00000100:00100000:2.0:1789760769.741914:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750959724143 time 751017294264 state 0x2 ref 0 00000100:00100000:3.0:1789760769.746879:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.747101:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317376 00000100:00100000:0.0:1789760769.747114:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317376 00000100:00100000:0.0:1789760769.747167:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 750959724143 time 751017294264 state 0x0 ref 1 00000100:00100000:0.0:1789760769.747182:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751017554680 time 751017294264 state 0x1 ref 1 00000100:00100000:0.0:1789760769.747219:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751017554680 time 751062522021 state 0x0 ref 1 00000100:00100000:0.0:1789760769.747231:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507317376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.747279:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21848064, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.747288:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.747302:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760769.747307:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.747313:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.747328:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.822878:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:2.0:1789760769.822912:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.822931:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.822943:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.822959:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760769.822966:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.822978:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.823007:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.823017:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b8e15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.823194:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507317376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75953us (76312us total) trans 4294972654 rc 0/0 00000100:00100000:2.0:1789760769.823211:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5364 00000100:00100000:2.0:1789760769.823392:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751017554680 time 751062522021 state 0x2 ref 0 00000001:00080000:3.0:1789760769.824557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00000100:00100000:0.0:1789760769.828946:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760769.829129:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317504 00000100:00100000:1.0:1789760769.829145:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317504 00000100:00100000:1.0:1789760769.829308:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751017554680 time 751062522021 state 0x0 ref 1 00000100:00100000:1.0:1789760769.829331:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751062782437 time 751062522021 state 0x1 ref 1 00000100:00100000:1.0:1789760769.829404:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751062782437 time 751144706205 state 0x0 ref 1 00000100:00100000:1.0:1789760769.829420:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507317504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760769.829499:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21852160, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760769.829511:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.829532:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760769.829538:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.829546:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.829564:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.888161:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00002000:00010000:2.0:1789760769.888389:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21852160 -> 21856256 00010000:00010000:2.0:1789760769.888409:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888424:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888439:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888446:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760769.888452:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888462:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888487:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.888495:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b8e1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.888800:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507317504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59374us (59862us total) trans 4294972655 rc 0/0 00000100:00100000:2.0:1789760769.888814:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5365 00000100:00100000:2.0:1789760769.888828:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751062782437 time 751144706205 state 0x2 ref 0 00000100:00100000:0.0:1789760769.892039:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760769.892161:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317632 00000100:00100000:2.0:1789760769.892175:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317632 00000100:00100000:2.0:1789760769.892410:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751062782437 time 751144706205 state 0x0 ref 1 00000100:00100000:2.0:1789760769.892431:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751144966621 time 751144706205 state 0x1 ref 1 00000100:00100000:2.0:1789760769.892508:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751144966621 time 751207810293 state 0x0 ref 1 00000100:00100000:2.0:1789760769.892524:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507317632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760769.892607:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21856256, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:2.0:1789760769.892627:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.892647:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760769.892653:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.892661:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.892678:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.941522:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972656 is committed 00002000:00010000:1.0:1789760769.941671:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21856256 -> 21860352 00010000:00010000:1.0:1789760769.941727:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941744:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941763:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941771:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760769.941777:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941788:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941814:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.941823:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b8e23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.941946:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507317632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49418us (49924us total) trans 4294972656 rc 0/0 00000100:00100000:1.0:1789760769.941959:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5366 00000100:00100000:1.0:1789760769.941970:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751144966621 time 751207810293 state 0x2 ref 0 00000100:00100000:2.0:1789760769.946023:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760769.946148:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317760 00000100:00100000:0.0:1789760769.946162:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317760 00000100:00100000:0.0:1789760769.946306:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751144966621 time 751207810293 state 0x0 ref 1 00000100:00100000:0.0:1789760769.946326:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751208070709 time 751207810293 state 0x1 ref 1 00000100:00100000:0.0:1789760769.946400:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751208070709 time 751261702654 state 0x0 ref 1 00000100:00100000:0.0:1789760769.946414:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760769.946480:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21860352, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:0.0:1789760769.946491:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.946518:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760769.946523:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.946531:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.946553:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.949048:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760769.997302:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:0.0:1789760769.997329:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997344:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997355:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997363:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760769.997369:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997380:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997405:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.997413:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8e2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.997629:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00000100:00100000:0.0:1789760769.998116:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51694us (52100us total) trans 4294972657 rc 0/0 00000100:00100000:0.0:1789760769.998130:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5367 00000100:00100000:0.0:1789760769.998143:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751208070709 time 751261702654 state 0x2 ref 0 00000100:00100000:2.0:1789760770.002791:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760770.002846:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507317888 00000100:00100000:1.0:1789760770.002861:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507317888 00000100:00100000:1.0:1789760770.002944:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751208070709 time 751261702654 state 0x0 ref 1 00000100:00100000:1.0:1789760770.002967:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751261963070 time 751261702654 state 0x1 ref 1 00000100:00100000:1.0:1789760770.003003:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751261963070 time 751318306139 state 0x0 ref 1 00000100:00100000:1.0:1789760770.003018:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760770.003129:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21864448, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xa807 00010000:00010000:1.0:1789760770.003141:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.003162:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760770.003167:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.003175:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.003192:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.039135:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00002000:00010000:2.0:1789760770.039331:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21864448 -> 21868544 00002000:00010000:2.0:1789760770.039337:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:2.0:1789760770.039340:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:2.0:1789760770.039357:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039372:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039383:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039389:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760770.039393:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039400:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039420:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.039427:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b8e31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.039533:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507317888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36512us (36750us total) trans 4294972658 rc 0/0 00000100:00100000:2.0:1789760770.039541:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5368 00000100:00100000:2.0:1789760770.039551:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751261963070 time 751318306139 state 0x2 ref 0 00000100:00100000:1.0:1789760770.046969:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.047148:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318016 00000100:00100000:2.0:1789760770.047291:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318016 00000100:00100000:2.0:1789760770.047371:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751261963070 time 751318306139 state 0x0 ref 1 00000100:00100000:2.0:1789760770.047392:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751318566555 time 751318306139 state 0x1 ref 1 00000100:00100000:2.0:1789760770.047433:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751318566555 time 751362736574 state 0x0 ref 1 00000100:00100000:2.0:1789760770.047447:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507318016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.047517:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21868544, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.047527:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.047548:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760770.047553:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.047560:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.047587:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.087729:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00002000:00010000:2.0:1789760770.087777:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:2.0:1789760770.087799:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087815:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087825:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087834:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760770.087841:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087852:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087879:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.087889:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.088014:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507318016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40562us (41055us total) trans 4294972659 rc 0/0 00000100:00100000:2.0:1789760770.088099:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5369 00000100:00100000:2.0:1789760770.088112:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751318566555 time 751362736574 state 0x2 ref 0 00000100:00100000:0.0:1789760770.095264:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.095544:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318144 00000100:00100000:2.0:1789760770.095558:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318144 00000100:00100000:2.0:1789760770.095777:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751318566555 time 751362736574 state 0x0 ref 1 00000100:00100000:2.0:1789760770.095797:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751362996990 time 751362736574 state 0x1 ref 1 00000100:00100000:2.0:1789760770.095863:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751362996990 time 751411165056 state 0x0 ref 1 00000100:00100000:2.0:1789760770.095879:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.095936:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21872640, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.095946:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.095963:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760770.095967:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.095972:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.095989:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.128765:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21872640 -> 21876736 00010000:00010000:3.0:1789760770.128800:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.128801:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00010000:00010000:3.0:1789760770.128815:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.128824:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.128833:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.128839:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.128850:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.128875:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.128890:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b8e3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.129070:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33186us (33813us total) trans 4294972660 rc 0/0 00000100:00100000:3.0:1789760770.129083:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5370 00000100:00100000:3.0:1789760770.129095:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751362996990 time 751411165056 state 0x2 ref 0 00000100:00100000:0.0:1789760770.132981:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.133179:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318272 00000100:00100000:2.0:1789760770.133192:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318272 00000100:00100000:2.0:1789760770.133746:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751362996990 time 751411165056 state 0x0 ref 1 00000100:00100000:2.0:1789760770.133766:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751411425472 time 751411165056 state 0x1 ref 1 00000100:00100000:2.0:1789760770.133795:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751411425472 time 751449098977 state 0x0 ref 1 00000100:00100000:2.0:1789760770.133811:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507318272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.133887:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21876736, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.133898:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.133919:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760770.133924:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.133933:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.133951:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.198833:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00002000:00010000:3.0:1789760770.199671:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21876736 -> 21880832 00010000:00010000:3.0:1789760770.199715:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199730:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199739:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199746:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.199752:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199764:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199796:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.199804:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b8e46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.199939:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507318272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66126us (66966us total) trans 4294972661 rc 0/0 00000100:00100000:3.0:1789760770.199951:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5371 00000100:00100000:3.0:1789760770.199963:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751411425472 time 751449098977 state 0x2 ref 0 00000100:00100000:1.0:1789760770.207100:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.207178:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318400 00000100:00100000:2.0:1789760770.207193:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318400 00000100:00100000:2.0:1789760770.207271:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751411425472 time 751449098977 state 0x0 ref 1 00000100:00100000:2.0:1789760770.207296:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751449359393 time 751449098977 state 0x1 ref 1 00000100:00100000:2.0:1789760770.207333:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751449359393 time 751522635410 state 0x0 ref 1 00000100:00100000:2.0:1789760770.207348:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.207420:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21880832, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.207432:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.207453:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760770.207458:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.207465:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.207483:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.239253:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00002000:00010000:0.0:1789760770.239329:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21880832 -> 21884928 00010000:00010000:0.0:1789760770.239349:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239364:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239372:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239377:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760770.239382:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239388:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239415:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.239420:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8e4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.239533:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32184us (32455us total) trans 4294972662 rc 0/0 00000100:00100000:0.0:1789760770.239542:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5372 00000100:00100000:0.0:1789760770.239553:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751449359393 time 751522635410 state 0x2 ref 0 00000100:00100000:0.0:1789760770.247388:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760770.247861:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318528 00000100:00100000:3.0:1789760770.247874:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318528 00000100:00100000:3.0:1789760770.247942:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751449359393 time 751522635410 state 0x0 ref 1 00000100:00100000:3.0:1789760770.247960:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751522895826 time 751522635410 state 0x1 ref 1 00000100:00100000:3.0:1789760770.247992:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751522895826 time 751563294945 state 0x0 ref 1 00000100:00100000:3.0:1789760770.248003:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507318528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.248062:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21884928, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.248072:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.248096:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760770.248101:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.248107:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.248123:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.267991:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538287232:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760770.268085:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760770.268230:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287232 00000100:00100000:0.0:1789760770.268242:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287232 00000100:00100000:0.0:1789760770.268312:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1789760770.268324:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538287232:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760770.268354:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3836739584 avail 3802136576 00000100:00100000:2.0:1789760770.268465:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538287232:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760770.268476:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538287232:12345-0@lo:13:osp-pre-0-1.0 Request processed in 155us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760770.268483:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1789760770.268895:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c2275ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538287360:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760770.268934:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760770.269027:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287360 00000100:00100000:3.0:1789760770.269031:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287360 00000100:00100000:3.0:1789760770.269097:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:3.0:1789760770.269104:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538287360:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760770.269119:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3836739584 avail 3802136576 00000100:00100000:3.0:1789760770.269160:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538287360:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760770.269167:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:3.0:1789760770.269278:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c2275ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538287360:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760770.316826:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:2.0:1789760770.316852:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316866:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316876:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316884:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760770.316890:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316900:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316925:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.316933:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b8e54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.317137:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507318528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69126us (69759us total) trans 4294972663 rc 0/0 00000100:00100000:2.0:1789760770.317150:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5373 00000100:00100000:2.0:1789760770.317162:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751522895826 time 751563294945 state 0x2 ref 0 00000001:00080000:0.0:1789760770.317366:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00000100:00100000:2.0:1789760770.323505:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760770.323622:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318656 00000100:00100000:3.0:1789760770.323636:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318656 00000100:00100000:3.0:1789760770.323853:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751522895826 time 751563294945 state 0x0 ref 1 00000100:00100000:3.0:1789760770.323878:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751563555361 time 751563294945 state 0x1 ref 1 00000100:00100000:3.0:1789760770.323905:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751563555361 time 751639208591 state 0x0 ref 1 00000100:00100000:3.0:1789760770.323921:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.323989:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21889024, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.324000:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.324152:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760770.324159:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.324167:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.324185:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.360172:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:3.0:1789760770.360301:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21889024 -> 21893120 00010000:00010000:3.0:1789760770.360322:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360338:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360348:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360363:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.360369:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360382:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360407:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.360416:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.360540:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36614us (37042us total) trans 4294972664 rc 0/0 00000100:00100000:3.0:1789760770.360551:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5374 00000100:00100000:3.0:1789760770.360563:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751563555361 time 751639208591 state 0x2 ref 0 00000100:00100000:3.0:1789760770.367879:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760770.368112:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318784 00000100:00100000:0.0:1789760770.368127:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318784 00000100:00100000:0.0:1789760770.368215:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751563555361 time 751639208591 state 0x0 ref 1 00000100:00100000:0.0:1789760770.368338:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751639469007 time 751639208591 state 0x1 ref 1 00000100:00100000:0.0:1789760770.368389:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751639469007 time 751683690410 state 0x0 ref 1 00000100:00100000:0.0:1789760770.368406:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760770.368502:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21893120, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:0.0:1789760770.368513:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.368535:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760770.368541:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.368549:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.368567:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.443402:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21893120 -> 21897216 00000001:00080000:1.0:1789760770.443457:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00010000:00010000:2.0:1789760770.443865:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.443879:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.443889:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.443895:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760770.443900:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.443941:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.443962:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.444035:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8e62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.444144:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75737us (76276us total) trans 4294972665 rc 0/0 00000100:00100000:2.0:1789760770.444154:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5375 00000100:00100000:2.0:1789760770.444163:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751639469007 time 751683690410 state 0x2 ref 0 00000100:00100000:0.0:1789760770.450088:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.459480:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507318912 00000100:00100000:2.0:1789760770.459567:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507318912 00000100:00100000:3.0:1789760770.463989:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751639469007 time 751683690410 state 0x0 ref 1 00000100:00100000:3.0:1789760770.464092:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751683950826 time 751683690410 state 0x1 ref 1 00000100:00100000:3.0:1789760770.464585:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751683950826 time 751779876852 state 0x0 ref 1 00000100:00100000:3.0:1789760770.464623:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.464748:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21897216, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.464759:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.464779:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760770.464785:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.464793:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.464813:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.524052:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:3.0:1789760770.524500:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:3.0:1789760770.524523:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.524535:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.524544:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.524551:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.524556:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.524565:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.524598:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.529361:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8e69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.529492:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507318912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64878us (79401us total) trans 4294972666 rc 0/0 00000100:00100000:3.0:1789760770.529503:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5376 00000100:00100000:3.0:1789760770.529512:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751683950826 time 751779876852 state 0x2 ref 0 00000100:00100000:0.0:1789760770.533202:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760770.533295:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319040 00000100:00100000:3.0:1789760770.533304:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319040 00000100:00100000:3.0:1789760770.533360:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751683950826 time 751779876852 state 0x0 ref 1 00000100:00100000:3.0:1789760770.533377:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751780137268 time 751779876852 state 0x1 ref 1 00000100:00100000:3.0:1789760770.533408:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751780137268 time 751848711555 state 0x0 ref 1 00000100:00100000:3.0:1789760770.533420:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.533467:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21901312, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.533477:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.533505:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760770.533510:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.533516:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.533530:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.597030:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00002000:00010000:3.0:1789760770.597418:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:3.0:1789760770.597433:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597445:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597453:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597460:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.597464:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597473:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597491:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.597498:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.597605:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64181us (64407us total) trans 4294972667 rc 0/0 00000100:00100000:3.0:1789760770.597623:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5377 00000100:00100000:3.0:1789760770.597633:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751780137268 time 751848711555 state 0x2 ref 0 00000100:00100000:1.0:1789760770.601135:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.601239:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319168 00000100:00100000:2.0:1789760770.601250:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319168 00000100:00100000:2.0:1789760770.601307:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751780137268 time 751848711555 state 0x0 ref 1 00000100:00100000:2.0:1789760770.601323:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751848971971 time 751848711555 state 0x1 ref 1 00000100:00100000:2.0:1789760770.601356:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751848971971 time 751916659631 state 0x0 ref 1 00000100:00100000:2.0:1789760770.601369:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.601419:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21905408, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.601428:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.601443:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760770.601448:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.601454:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.601469:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.642397:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00002000:00010000:0.0:1789760770.642490:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21905408 -> 21909504 00010000:00010000:0.0:1789760770.642505:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642519:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642528:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642536:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760770.642541:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642550:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642578:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.642593:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b8e77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.643164:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41791us (42032us total) trans 4294972668 rc 0/0 00000100:00100000:0.0:1789760770.643175:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5378 00000100:00100000:0.0:1789760770.643185:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751848971971 time 751916659631 state 0x2 ref 0 00000100:00100000:0.0:1789760770.651374:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760770.651450:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319296 00000100:00100000:3.0:1789760770.651461:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319296 00000100:00100000:3.0:1789760770.651514:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751848971971 time 751916659631 state 0x0 ref 1 00000100:00100000:3.0:1789760770.651530:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751916920047 time 751916659631 state 0x1 ref 1 00000100:00100000:3.0:1789760770.651559:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751916920047 time 751966863038 state 0x0 ref 1 00000100:00100000:3.0:1789760770.651581:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507319296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.651645:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21909504, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.651654:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.651670:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760770.651674:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.651680:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.651758:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.717105:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:3.0:1789760770.717123:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717135:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717143:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717150:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.717155:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717164:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717184:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.717192:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.717291:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507319296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65717us (65925us total) trans 4294972669 rc 0/0 00000100:00100000:3.0:1789760770.717301:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5379 00000100:00100000:3.0:1789760770.717309:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751916920047 time 751966863038 state 0x2 ref 0 00000001:00080000:1.0:1789760770.718808:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00000100:00100000:1.0:1789760770.727113:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760770.727197:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319424 00000100:00100000:3.0:1789760770.727206:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319424 00000100:00100000:3.0:1789760770.727256:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751916920047 time 751966863038 state 0x0 ref 1 00000100:00100000:3.0:1789760770.727271:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751967123454 time 751966863038 state 0x1 ref 1 00000100:00100000:3.0:1789760770.727301:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751967123454 time 752042605009 state 0x0 ref 1 00000100:00100000:3.0:1789760770.727312:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507319424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760770.727360:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21913600, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:3.0:1789760770.727368:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.727384:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760770.727388:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.727394:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.727406:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.782112:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760770.782136:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538287488:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760770.782202:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760770.782408:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287488 00000100:00100000:0.0:1789760770.782417:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287488 00000100:00100000:0.0:1789760770.782469:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:0.0:1789760770.782480:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538287488:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760770.782686:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760770.782771:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538287488:12345-0@lo:41:osp-pre-1-0.0 Request processed in 292us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760770.782780:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789760770.782861:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538287488:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789760770.793038:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00002000:00010000:3.0:1789760770.793340:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:3.0:1789760770.793354:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793366:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793374:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793381:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760770.793385:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793393:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793411:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.793418:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8e85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.793536:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507319424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66221us (66428us total) trans 4294972670 rc 0/0 00000100:00100000:3.0:1789760770.793546:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5380 00000100:00100000:3.0:1789760770.793556:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751967123454 time 752042605009 state 0x2 ref 0 00000100:00100000:0.0:1789760770.800550:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760770.800651:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319552 00000100:00100000:2.0:1789760770.800660:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319552 00000100:00100000:2.0:1789760770.800739:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 751967123454 time 752042605009 state 0x0 ref 1 00000100:00100000:2.0:1789760770.800753:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752042865425 time 752042605009 state 0x1 ref 1 00000100:00100000:2.0:1789760770.800773:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752042865425 time 752116076537 state 0x0 ref 1 00000100:00100000:2.0:1789760770.800784:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760770.800828:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21917696, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:2.0:1789760770.800837:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.800852:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760770.800856:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.800862:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.800875:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.843924:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:2.0:1789760770.843943:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.843971:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.843982:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.843989:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760770.843994:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.844178:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.844202:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.844209:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b8e8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.844309:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507319552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43521us (43762us total) trans 4294972671 rc 0/0 00000100:00100000:2.0:1789760770.844319:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5381 00000100:00100000:2.0:1789760770.844328:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752042865425 time 752116076537 state 0x2 ref 0 00000001:00080000:1.0:1789760770.844409:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00000100:00100000:3.0:1789760770.848161:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760770.848317:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319680 00000100:00100000:1.0:1789760770.848328:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319680 00000100:00100000:1.0:1789760770.848385:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752042865425 time 752116076537 state 0x0 ref 1 00000100:00100000:1.0:1789760770.848400:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752116336953 time 752116076537 state 0x1 ref 1 00000100:00100000:1.0:1789760770.848425:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752116336953 time 752163728431 state 0x0 ref 1 00000100:00100000:1.0:1789760770.848437:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760770.848485:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21921792, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:1.0:1789760770.848493:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.848509:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760770.848513:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.848519:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.848533:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.892108:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00002000:00010000:1.0:1789760770.892210:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:1.0:1789760770.892226:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892240:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892251:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892258:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760770.892262:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892272:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892291:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.892300:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8e93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.892392:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43952us (44235us total) trans 4294972672 rc 0/0 00000100:00100000:1.0:1789760770.892402:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5382 00000100:00100000:1.0:1789760770.892412:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752116336953 time 752163728431 state 0x2 ref 0 00000100:00100000:3.0:1789760770.896378:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760770.896949:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319808 00000100:00100000:1.0:1789760770.896959:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319808 00000100:00100000:1.0:1789760770.897223:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752116336953 time 752163728431 state 0x0 ref 1 00000100:00100000:1.0:1789760770.897240:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752163988847 time 752163728431 state 0x1 ref 1 00000100:00100000:1.0:1789760770.897303:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752163988847 time 752212605780 state 0x0 ref 1 00000100:00100000:1.0:1789760770.897317:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507319808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760770.897360:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21925888, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:1.0:1789760770.897369:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897384:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760770.897388:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.897394:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897407:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.945864:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:1.0:1789760770.945883:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945897:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945907:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945915:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760770.945920:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945929:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945948:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.945955:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8e9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.946050:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507319808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48731us (49680us total) trans 4294972673 rc 0/0 00000100:00100000:1.0:1789760770.946061:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5383 00000100:00100000:1.0:1789760770.946070:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752163988847 time 752212605780 state 0x2 ref 0 00000001:00080000:3.0:1789760770.946884:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00000100:00100000:2.0:1789760770.953141:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760770.953393:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507319936 00000100:00100000:0.0:1789760770.953402:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507319936 00000100:00100000:0.0:1789760770.953458:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752163988847 time 752212605780 state 0x0 ref 1 00000100:00100000:0.0:1789760770.953474:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752212866196 time 752212605780 state 0x1 ref 1 00000100:00100000:0.0:1789760770.953504:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752212866196 time 752268807890 state 0x0 ref 1 00000100:00100000:0.0:1789760770.953516:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507319936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760770.953562:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21929984, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:0.0:1789760770.953583:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.953598:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760770.953610:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.953617:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.953630:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.974423:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760770.986507:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00002000:00010000:1.0:1789760770.986636:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:1.0:1789760770.986652:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986663:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986672:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986683:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760770.986687:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986726:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986746:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.986753:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8ea1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.986835:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507319936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33315us (33699us total) trans 4294972674 rc 0/0 00000100:00100000:1.0:1789760770.986844:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5384 00000100:00100000:1.0:1789760770.986854:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752212866196 time 752268807890 state 0x2 ref 0 00000100:00100000:0.0:1789760770.995041:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760770.995132:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320064 00000100:00100000:1.0:1789760770.995142:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320064 00000100:00100000:1.0:1789760770.995207:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752212866196 time 752268807890 state 0x0 ref 1 00000100:00100000:1.0:1789760770.995223:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752269068306 time 752268807890 state 0x1 ref 1 00000100:00100000:1.0:1789760770.995252:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752269068306 time 752310555378 state 0x0 ref 1 00000100:00100000:1.0:1789760770.995268:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760770.995315:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21934080, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xa807 00010000:00010000:1.0:1789760770.995324:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.995340:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760770.995344:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.995351:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.995365:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.033396:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00002000:00010000:1.0:1789760771.033497:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21934080 -> 21938176 00002000:00010000:1.0:1789760771.033501:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:1.0:1789760771.033504:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:1.0:1789760771.033516:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033530:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033539:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033547:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760771.033552:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033561:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033593:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.033600:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8ea8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.033765:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38495us (38728us total) trans 4294972675 rc 0/0 00000100:00100000:1.0:1789760771.033775:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5385 00000100:00100000:1.0:1789760771.033784:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752269068306 time 752310555378 state 0x2 ref 0 00000100:00100000:2.0:1789760771.047846:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.047976:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320192 00000100:00100000:1.0:1789760771.047987:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320192 00000100:00100000:1.0:1789760771.048067:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752269068306 time 752310555378 state 0x0 ref 1 00000100:00100000:1.0:1789760771.048084:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752310815794 time 752310555378 state 0x1 ref 1 00000100:00100000:1.0:1789760771.048125:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752310815794 time 752363427560 state 0x0 ref 1 00000100:00100000:1.0:1789760771.048139:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320192:12345-192.168.203.41@tcp:4:dd.0 00000100:00080000:0.0:1789760771.048313:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760771.048438:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21938176, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.048447:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.048463:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.048468:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.048473:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.048487:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.087025:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00002000:00010000:0.0:1789760771.087112:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:0.0:1789760771.087127:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087140:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087150:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087158:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760771.087163:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087238:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087259:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.087267:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b8eaf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.087396:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39255us (39555us total) trans 4294972676 rc 0/0 00000100:00100000:0.0:1789760771.087406:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5386 00000100:00100000:0.0:1789760771.087415:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752310815794 time 752363427560 state 0x2 ref 0 00000100:00100000:3.0:1789760771.090937:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.091151:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320320 00000100:00100000:1.0:1789760771.091161:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320320 00000100:00100000:1.0:1789760771.091221:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752310815794 time 752363427560 state 0x0 ref 1 00000100:00100000:1.0:1789760771.091238:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752363687976 time 752363427560 state 0x1 ref 1 00000100:00100000:1.0:1789760771.091267:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752363687976 time 752406570318 state 0x0 ref 1 00000100:00100000:1.0:1789760771.091278:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507320320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760771.091322:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21942272, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.091330:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.091345:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.091350:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.091356:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.091370:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.121679:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:2.0:1789760771.122099:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:2.0:1789760771.122117:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122130:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122139:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122147:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.122151:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122160:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122178:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.122186:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b8eb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.122321:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507320320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31039us (31389us total) trans 4294972677 rc 0/0 00000100:00100000:2.0:1789760771.122332:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5387 00000100:00100000:2.0:1789760771.122342:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752363687976 time 752406570318 state 0x2 ref 0 00000100:00100000:3.0:1789760771.126526:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.126864:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320448 00000100:00100000:1.0:1789760771.126875:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320448 00000100:00100000:1.0:1789760771.126927:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752363687976 time 752406570318 state 0x0 ref 1 00000100:00100000:1.0:1789760771.126943:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752406830734 time 752406570318 state 0x1 ref 1 00000100:00100000:1.0:1789760771.126976:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752406830734 time 752442278747 state 0x0 ref 1 00000100:00100000:1.0:1789760771.126989:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760771.127047:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21946368, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.127056:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.127072:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.127077:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.127084:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.127099:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.157286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00002000:00010000:1.0:1789760771.157395:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:1.0:1789760771.157410:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157423:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157432:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157439:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760771.157444:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157453:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157474:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.157482:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ebd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.157617:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30627us (31097us total) trans 4294972678 rc 0/0 00000100:00100000:1.0:1789760771.157627:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5388 00000100:00100000:1.0:1789760771.157637:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752406830734 time 752442278747 state 0x2 ref 0 00000100:00100000:2.0:1789760771.161320:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760771.161402:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320576 00000100:00100000:0.0:1789760771.161413:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320576 00000100:00100000:0.0:1789760771.161473:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752406830734 time 752442278747 state 0x0 ref 1 00000100:00100000:0.0:1789760771.161489:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752442539163 time 752442278747 state 0x1 ref 1 00000100:00100000:0.0:1789760771.161520:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752442539163 time 752476822832 state 0x0 ref 1 00000100:00100000:0.0:1789760771.161533:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507320576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760771.161592:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21950464, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:0.0:1789760771.161889:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161906:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760771.161911:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.161916:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161930:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.191554:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00002000:00010000:1.0:1789760771.191736:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:1.0:1789760771.191753:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191766:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191776:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191783:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760771.191789:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191798:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191818:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.191826:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b8ec4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.191965:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507320576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30430us (30650us total) trans 4294972679 rc 0/0 00000100:00100000:1.0:1789760771.191976:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5389 00000100:00100000:1.0:1789760771.191986:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752442539163 time 752476822832 state 0x2 ref 0 00000100:00100000:0.0:1789760771.195481:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.195591:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320704 00000100:00100000:2.0:1789760771.195609:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320704 00000100:00100000:2.0:1789760771.195663:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752442539163 time 752476822832 state 0x0 ref 1 00000100:00100000:2.0:1789760771.195679:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752477083248 time 752476822832 state 0x1 ref 1 00000100:00100000:2.0:1789760771.195836:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752477083248 time 752511139316 state 0x0 ref 1 00000100:00100000:2.0:1789760771.195846:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.195888:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21954560, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.195896:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.195910:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760771.195914:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.195920:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.195932:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.221224:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00002000:00010000:2.0:1789760771.221376:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21954560 -> 21958656 00010000:00010000:2.0:1789760771.221392:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221405:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221413:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221420:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.221425:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221433:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221454:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.221462:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ecb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.221544:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25694us (26073us total) trans 4294972680 rc 0/0 00000100:00100000:2.0:1789760771.221554:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5390 00000100:00100000:2.0:1789760771.221564:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752477083248 time 752511139316 state 0x2 ref 0 00000100:00100000:0.0:1789760771.225796:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.225940:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320832 00000100:00100000:1.0:1789760771.225948:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320832 00000100:00100000:1.0:1789760771.226006:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752477083248 time 752511139316 state 0x0 ref 1 00000100:00100000:1.0:1789760771.226021:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752511399732 time 752511139316 state 0x1 ref 1 00000100:00100000:1.0:1789760771.226053:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752511399732 time 752541356204 state 0x0 ref 1 00000100:00100000:1.0:1789760771.226066:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507320832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760771.226112:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21958656, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.226119:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.226132:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.226135:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.226139:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.226149:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.251183:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21958656 -> 21962752 00010000:00010000:1.0:1789760771.251206:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251225:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251235:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251243:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760771.251249:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251258:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251276:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.251283:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.251377:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507320832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25309us (25584us total) trans 4294972681 rc 0/0 00000100:00100000:1.0:1789760771.251388:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5391 00000100:00100000:1.0:1789760771.251398:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752511399732 time 752541356204 state 0x2 ref 0 00000001:00080000:0.0:1789760771.252043:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00000100:00100000:3.0:1789760771.257487:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.257670:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507320960 00000100:00100000:1.0:1789760771.257682:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507320960 00000100:00100000:1.0:1789760771.257767:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752511399732 time 752541356204 state 0x0 ref 1 00000100:00100000:1.0:1789760771.257782:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752541616620 time 752541356204 state 0x1 ref 1 00000100:00100000:1.0:1789760771.257901:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752541616620 time 752573203090 state 0x0 ref 1 00000100:00100000:1.0:1789760771.257914:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760771.257960:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21962752, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.257970:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257983:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.257988:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.257994:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.258008:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.293065:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760771.293085:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b701180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538287616:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760771.293150:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.293215:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287616 00000100:00100000:3.0:1789760771.293221:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287616 00000100:00100000:3.0:1789760771.293262:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:3.0:1789760771.293271:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538287616:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760771.293305:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760771.293354:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538287616:12345-0@lo:41:osp-pre-0-1.0 Request processed in 83us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.293361:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:3.0:1789760771.293406:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b701180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538287616:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789760771.300630:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00002000:00010000:0.0:1789760771.300823:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21962752 -> 21966848 00010000:00010000:0.0:1789760771.300839:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300851:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300861:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300869:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760771.300873:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300882:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300902:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.300909:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b8ed9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.301038:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507320960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43121us (43557us total) trans 4294972682 rc 0/0 00000100:00100000:0.0:1789760771.301049:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5392 00000100:00100000:0.0:1789760771.301058:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752541616620 time 752573203090 state 0x2 ref 0 00000100:00100000:3.0:1789760771.310900:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.313666:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321088 00000100:00100000:2.0:1789760771.313681:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321088 00000100:00100000:2.0:1789760771.313758:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752541616620 time 752573203090 state 0x0 ref 1 00000100:00100000:2.0:1789760771.313774:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752573463506 time 752573203090 state 0x1 ref 1 00000100:00100000:2.0:1789760771.313863:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752573463506 time 752629166164 state 0x0 ref 1 00000100:00100000:2.0:1789760771.313879:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507321088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.313926:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21966848, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.313935:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.313950:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760771.313954:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.313962:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.313975:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.361463:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:2.0:1789760771.361683:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:2.0:1789760771.361725:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361739:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361748:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361755:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.361760:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361769:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361789:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.361797:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ee0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.361893:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507321088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48014us (51000us total) trans 4294972683 rc 0/0 00000100:00100000:2.0:1789760771.361904:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5393 00000100:00100000:2.0:1789760771.361913:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752573463506 time 752629166164 state 0x2 ref 0 00000100:00100000:3.0:1789760771.371730:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760771.371864:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321216 00000100:00100000:1.0:1789760771.371875:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321216 00000100:00100000:1.0:1789760771.371931:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752573463506 time 752629166164 state 0x0 ref 1 00000100:00100000:1.0:1789760771.371947:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752629426580 time 752629166164 state 0x1 ref 1 00000100:00100000:1.0:1789760771.371976:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752629426580 time 752687279120 state 0x0 ref 1 00000100:00100000:1.0:1789760771.371993:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507321216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760771.372053:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21970944, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:1.0:1789760771.372063:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.372079:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760771.372084:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.372091:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.372105:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.410907:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:2.0:1789760771.410927:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.410947:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.410958:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.410966:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.410970:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.410980:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.411002:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.411011:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b8ee7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.411252:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507321216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39255us (39556us total) trans 4294972684 rc 0/0 00000100:00100000:2.0:1789760771.411264:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5394 00000100:00100000:2.0:1789760771.411275:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752629426580 time 752687279120 state 0x2 ref 0 00000001:00080000:3.0:1789760771.411455:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00000100:00100000:3.0:1789760771.414888:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760771.414948:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321344 00000100:00100000:0.0:1789760771.414959:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321344 00000100:00100000:0.0:1789760771.415014:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752629426580 time 752687279120 state 0x0 ref 1 00000100:00100000:0.0:1789760771.415030:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752687539536 time 752687279120 state 0x1 ref 1 00000100:00100000:0.0:1789760771.415206:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752687539536 time 752730507263 state 0x0 ref 1 00000100:00100000:0.0:1789760771.415220:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507321344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760771.415268:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21975040, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:0.0:1789760771.415277:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.415293:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760771.415298:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.415304:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.415322:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.474734:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00002000:00010000:0.0:1789760771.474822:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:0.0:1789760771.474835:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474845:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474853:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474857:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760771.474861:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474866:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474880:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.474885:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b8eee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.474966:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507321344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59744us (60083us total) trans 4294972685 rc 0/0 00000100:00100000:0.0:1789760771.474976:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5395 00000100:00100000:0.0:1789760771.474985:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752687539536 time 752730507263 state 0x2 ref 0 00000100:00100000:0.0:1789760771.479237:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.479452:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321472 00000100:00100000:2.0:1789760771.479463:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321472 00000100:00100000:2.0:1789760771.479517:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752687539536 time 752730507263 state 0x0 ref 1 00000100:00100000:2.0:1789760771.479532:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752730767679 time 752730507263 state 0x1 ref 1 00000100:00100000:2.0:1789760771.479608:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752730767679 time 752794911013 state 0x0 ref 1 00000100:00100000:2.0:1789760771.479622:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507321472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.479667:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21979136, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.479676:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.479724:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760771.479729:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.479735:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.479751:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.525518:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21979136 -> 21983232 00010000:00010000:2.0:1789760771.525538:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525550:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525574:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525581:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.525586:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525595:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525613:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.525621:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8ef5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.525745:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507321472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46120us (46518us total) trans 4294972686 rc 0/0 00000100:00100000:2.0:1789760771.525756:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5396 00000100:00100000:2.0:1789760771.525767:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752730767679 time 752794911013 state 0x2 ref 0 00000001:00080000:0.0:1789760771.525995:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00000100:00100000:3.0:1789760771.532020:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.532107:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321600 00000100:00100000:2.0:1789760771.532117:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321600 00000100:00100000:2.0:1789760771.532172:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752730767679 time 752794911013 state 0x0 ref 1 00000100:00100000:2.0:1789760771.532187:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752795171429 time 752794911013 state 0x1 ref 1 00000100:00100000:2.0:1789760771.532221:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752795171429 time 752847524436 state 0x0 ref 1 00000100:00100000:2.0:1789760771.532234:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507321600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.532285:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21983232, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.532294:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.532309:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760771.532314:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.532321:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.532336:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.561373:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00002000:00010000:2.0:1789760771.561686:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:2.0:1789760771.561717:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561730:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561739:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561746:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.561751:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561761:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561782:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.561789:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8efc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.561920:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507321600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29683us (29905us total) trans 4294972687 rc 0/0 00000100:00100000:2.0:1789760771.561931:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5397 00000100:00100000:2.0:1789760771.561940:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752795171429 time 752847524436 state 0x2 ref 0 00000100:00100000:1.0:1789760771.572513:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760771.572677:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321728 00000100:00100000:0.0:1789760771.572688:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321728 00000100:00100000:0.0:1789760771.572854:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752795171429 time 752847524436 state 0x0 ref 1 00000100:00100000:0.0:1789760771.572871:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752847784852 time 752847524436 state 0x1 ref 1 00000100:00100000:0.0:1789760771.572926:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752847784852 time 752888229213 state 0x0 ref 1 00000100:00100000:0.0:1789760771.572939:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507321728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760771.572980:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21987328, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:0.0:1789760771.572990:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.573005:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760771.573010:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.573025:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.573040:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.624887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:0.0:1789760771.624953:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:0.0:1789760771.624968:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624981:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624996:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.625004:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760771.625009:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.625018:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.625049:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.625058:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b8f03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.625222:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507321728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52279us (52712us total) trans 4294972688 rc 0/0 00000100:00100000:0.0:1789760771.625233:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5398 00000100:00100000:0.0:1789760771.625243:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752847784852 time 752888229213 state 0x2 ref 0 00000100:00100000:0.0:1789760771.640577:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.640733:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321856 00000100:00100000:2.0:1789760771.640744:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321856 00000100:00100000:2.0:1789760771.640800:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752847784852 time 752888229213 state 0x0 ref 1 00000100:00100000:2.0:1789760771.640815:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752888489629 time 752888229213 state 0x1 ref 1 00000100:00100000:2.0:1789760771.640845:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752888489629 time 752956148715 state 0x0 ref 1 00000100:00100000:2.0:1789760771.640858:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507321856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.640906:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21991424, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.640916:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.640932:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760771.640937:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.640942:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.640955:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.704204:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:0.0:1789760771.704223:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704244:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704252:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704259:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760771.704264:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704274:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704295:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.704302:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.704401:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507321856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63539us (63839us total) trans 4294972689 rc 0/0 00000100:00100000:0.0:1789760771.704412:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5399 00000100:00100000:0.0:1789760771.704421:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752888489629 time 752956148715 state 0x2 ref 0 00000001:00080000:1.0:1789760771.705472:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00000100:00100000:3.0:1789760771.716816:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.716998:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507321984 00000100:00100000:2.0:1789760771.717007:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507321984 00000100:00100000:2.0:1789760771.717068:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752888489629 time 752956148715 state 0x0 ref 1 00000100:00100000:2.0:1789760771.717084:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752956409131 time 752956148715 state 0x1 ref 1 00000100:00100000:2.0:1789760771.717119:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752956409131 time 753032422387 state 0x0 ref 1 00000100:00100000:2.0:1789760771.717132:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507321984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.717182:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21995520, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.717192:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.717208:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760771.717213:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.717218:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.717234:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.759816:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:2.0:1789760771.760126:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:2.0:1789760771.760143:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760154:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760230:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760239:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.760243:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760251:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760271:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.760278:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.760375:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507321984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43239us (43563us total) trans 4294972690 rc 0/0 00000100:00100000:2.0:1789760771.760384:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5400 00000100:00100000:2.0:1789760771.760393:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752956409131 time 753032422387 state 0x2 ref 0 00000100:00100000:2.0:1789760771.763512:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760771.763635:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322112 00000100:00100000:0.0:1789760771.763645:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322112 00000100:00100000:0.0:1789760771.763734:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 752956409131 time 753032422387 state 0x0 ref 1 00000100:00100000:0.0:1789760771.763749:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753032682803 time 753032422387 state 0x1 ref 1 00000100:00100000:0.0:1789760771.763778:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753032682803 time 753079081389 state 0x0 ref 1 00000100:00100000:0.0:1789760771.763789:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507322112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760771.763833:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21999616, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:0.0:1789760771.763841:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.763855:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760771.763860:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.763865:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.763879:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.804337:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00002000:00010000:3.0:1789760771.804571:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21999616 -> 22003712 00010000:00010000:3.0:1789760771.804587:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804598:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804607:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804614:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760771.804619:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804627:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804652:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.804660:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b8f18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.804827:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507322112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41035us (41319us total) trans 4294972691 rc 0/0 00000100:00100000:3.0:1789760771.804837:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5401 00000100:00100000:3.0:1789760771.804846:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753032682803 time 753079081389 state 0x2 ref 0 00000100:00100000:0.0:1789760771.807930:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.808025:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322240 00000100:00100000:2.0:1789760771.808034:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322240 00000100:00100000:2.0:1789760771.808088:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753032682803 time 753079081389 state 0x0 ref 1 00000100:00100000:2.0:1789760771.808101:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753079341805 time 753079081389 state 0x1 ref 1 00000100:00100000:2.0:1789760771.808134:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753079341805 time 753123437349 state 0x0 ref 1 00000100:00100000:2.0:1789760771.808147:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507322240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.808302:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22003712, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.808312:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.808327:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760771.808332:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.808338:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.808352:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.863493:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:2.0:1789760771.863555:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22003712 -> 22007808 00010000:00010000:2.0:1789760771.863583:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863595:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863670:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863678:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.863683:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863822:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863844:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.863852:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.864155:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507322240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56005us (56229us total) trans 4294972692 rc 0/0 00000100:00100000:2.0:1789760771.864168:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5402 00000100:00100000:2.0:1789760771.864178:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753079341805 time 753123437349 state 0x2 ref 0 00000100:00100000:3.0:1789760771.869265:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.869347:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322368 00000100:00100000:2.0:1789760771.869357:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322368 00000100:00100000:2.0:1789760771.869410:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753079341805 time 753123437349 state 0x0 ref 1 00000100:00100000:2.0:1789760771.869425:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753123697765 time 753123437349 state 0x1 ref 1 00000100:00100000:2.0:1789760771.869454:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753123697765 time 753184756923 state 0x0 ref 1 00000100:00100000:2.0:1789760771.869466:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507322368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.869514:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22007808, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.869524:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.869538:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760771.869542:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.869548:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.869571:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.926509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00002000:00010000:2.0:1789760771.926993:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22007808 -> 22011904 00010000:00010000:2.0:1789760771.927083:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927098:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927107:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927115:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760771.927120:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927130:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927227:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.927237:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.927378:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507322368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57909us (58119us total) trans 4294972693 rc 0/0 00000100:00100000:2.0:1789760771.927390:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5403 00000100:00100000:2.0:1789760771.927400:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753123697765 time 753184756923 state 0x2 ref 0 00000100:00100000:1.0:1789760771.938336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760771.938943:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322496 00000100:00100000:3.0:1789760771.938954:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322496 00000100:00100000:3.0:1789760771.939009:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753123697765 time 753184756923 state 0x0 ref 1 00000100:00100000:3.0:1789760771.939024:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753185017339 time 753184756923 state 0x1 ref 1 00000100:00100000:3.0:1789760771.939103:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753185017339 time 753254405826 state 0x0 ref 1 00000100:00100000:3.0:1789760771.939116:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507322496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760771.939177:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22011904, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:3.0:1789760771.939187:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.939208:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760771.939213:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.939219:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.939234:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.985623:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:3.0:1789760771.985646:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985661:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985670:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985677:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760771.985682:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985721:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985742:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.985760:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.985950:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00000100:00100000:3.0:1789760771.986192:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507322496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46736us (47525us total) trans 4294972694 rc 0/0 00000100:00100000:3.0:1789760771.986205:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5404 00000100:00100000:3.0:1789760771.986215:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753185017339 time 753254405826 state 0x2 ref 0 00000100:00100000:0.0:1789760771.991423:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760771.991552:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322624 00000100:00100000:2.0:1789760771.991572:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322624 00000100:00100000:2.0:1789760771.991631:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753185017339 time 753254405826 state 0x0 ref 1 00000100:00100000:2.0:1789760771.991646:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753254666242 time 753254405826 state 0x1 ref 1 00000100:00100000:2.0:1789760771.991727:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753254666242 time 753307029181 state 0x0 ref 1 00000100:00100000:2.0:1789760771.991741:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507322624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760771.991973:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22016000, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xa807 00010000:00010000:2.0:1789760771.991983:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.991997:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760771.992001:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.992007:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.992021:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.995815:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760772.030339:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:2.0:1789760772.030509:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22016000 -> 22020096 00002000:00010000:2.0:1789760772.030513:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:2.0:1789760772.030516:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:2.0:1789760772.030530:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030549:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030558:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030576:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760772.030581:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030590:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030620:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.030628:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.030744:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507322624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39000us (39326us total) trans 4294972695 rc 0/0 00000100:00100000:2.0:1789760772.030754:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5405 00000100:00100000:2.0:1789760772.031967:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753254666242 time 753307029181 state 0x2 ref 0 00000100:00100000:0.0:1789760772.038841:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.038930:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322752 00000100:00100000:2.0:1789760772.038940:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322752 00000100:00100000:2.0:1789760772.038994:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753254666242 time 753307029181 state 0x0 ref 1 00000100:00100000:2.0:1789760772.039009:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753307289597 time 753307029181 state 0x1 ref 1 00000100:00100000:2.0:1789760772.039045:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753307289597 time 753354346992 state 0x0 ref 1 00000100:00100000:2.0:1789760772.039057:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507322752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.039105:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22020096, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xa807 00010000:00010000:2.0:1789760772.039114:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.039130:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.039134:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.039140:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.039155:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.062478:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00002000:00010000:0.0:1789760772.062537:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:0.0:1789760772.062542:0:8092:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:0.0:1789760772.062556:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062585:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062594:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062601:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.062606:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062615:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062635:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.062644:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.062781:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507322752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23722us (23946us total) trans 4294972696 rc 0/0 00000100:00100000:0.0:1789760772.062791:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5406 00000100:00100000:0.0:1789760772.062800:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753307289597 time 753354346992 state 0x2 ref 0 00000100:00100000:1.0:1789760772.068956:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760772.069142:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507322880 00000100:00100000:3.0:1789760772.069151:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507322880 00000100:00100000:3.0:1789760772.069200:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753307289597 time 753354346992 state 0x0 ref 1 00000100:00100000:3.0:1789760772.069214:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753354607408 time 753354346992 state 0x1 ref 1 00000100:00100000:3.0:1789760772.069247:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753354607408 time 753384550762 state 0x0 ref 1 00000100:00100000:3.0:1789760772.069258:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507322880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760772.069313:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22024192, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:3.0:1789760772.069321:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.069336:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760772.069342:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.069348:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.069362:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.118011:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00002000:00010000:2.0:1789760772.118191:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:2.0:1789760772.118207:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118219:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118234:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118242:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760772.118248:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118257:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118277:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.118285:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b8f42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.118414:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507322880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49151us (49462us total) trans 4294972697 rc 0/0 00000100:00100000:2.0:1789760772.118424:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5407 00000100:00100000:2.0:1789760772.118433:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753354607408 time 753384550762 state 0x2 ref 0 00000100:00100000:3.0:1789760772.122645:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.122833:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323008 00000100:00100000:2.0:1789760772.122843:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323008 00000100:00100000:2.0:1789760772.122907:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753354607408 time 753384550762 state 0x0 ref 1 00000100:00100000:2.0:1789760772.122922:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753384811178 time 753384550762 state 0x1 ref 1 00000100:00100000:2.0:1789760772.122955:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753384811178 time 753438258131 state 0x0 ref 1 00000100:00100000:2.0:1789760772.122967:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.123280:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22028288, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.123297:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.123312:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.123317:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.123322:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.123335:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.174447:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00002000:00010000:0.0:1789760772.174534:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:0.0:1789760772.174550:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174570:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174578:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174584:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.174588:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174596:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174756:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.174764:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b8f49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.174846:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51876us (52207us total) trans 4294972698 rc 0/0 00000100:00100000:0.0:1789760772.174857:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5408 00000100:00100000:0.0:1789760772.174867:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753384811178 time 753438258131 state 0x2 ref 0 00000100:00100000:0.0:1789760772.179993:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.180099:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323136 00000100:00100000:2.0:1789760772.180108:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323136 00000100:00100000:2.0:1789760772.180167:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753384811178 time 753438258131 state 0x0 ref 1 00000100:00100000:2.0:1789760772.180182:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753438518547 time 753438258131 state 0x1 ref 1 00000100:00100000:2.0:1789760772.180210:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753438518547 time 753495513809 state 0x0 ref 1 00000100:00100000:2.0:1789760772.180222:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507323136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.180271:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22032384, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.180281:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.180300:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.180304:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.180310:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.180323:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.210541:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:0.0:1789760772.210569:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210583:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210592:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210608:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.210613:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210622:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210642:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.210649:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.210774:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507323136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30549us (30785us total) trans 4294972699 rc 0/0 00000100:00100000:0.0:1789760772.210788:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5409 00000100:00100000:0.0:1789760772.210798:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753438518547 time 753495513809 state 0x2 ref 0 00000001:00080000:3.0:1789760772.211137:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00000100:00100000:1.0:1789760772.215888:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.215974:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323264 00000100:00100000:2.0:1789760772.215983:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323264 00000100:00100000:2.0:1789760772.216037:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753438518547 time 753495513809 state 0x0 ref 1 00000100:00100000:2.0:1789760772.216052:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753495774225 time 753495513809 state 0x1 ref 1 00000100:00100000:2.0:1789760772.216081:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753495774225 time 753531384089 state 0x0 ref 1 00000100:00100000:2.0:1789760772.216093:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.216141:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22036480, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.216150:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.216165:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.216184:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.216191:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.216205:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.243194:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00002000:00010000:3.0:1789760772.243202:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:3.0:1789760772.243217:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243230:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243239:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243246:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760772.243250:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243257:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243272:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.243280:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b8f57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.243356:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27260us (27473us total) trans 4294972700 rc 0/0 00000100:00100000:3.0:1789760772.243366:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5410 00000100:00100000:3.0:1789760772.243375:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753495774225 time 753531384089 state 0x2 ref 0 00000100:00100000:3.0:1789760772.246344:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.246477:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323392 00000100:00100000:0.0:1789760772.246487:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323392 00000100:00100000:0.0:1789760772.246536:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753495774225 time 753531384089 state 0x0 ref 1 00000100:00100000:0.0:1789760772.246550:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753531644505 time 753531384089 state 0x1 ref 1 00000100:00100000:0.0:1789760772.246618:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753531644505 time 753561921111 state 0x0 ref 1 00000100:00100000:0.0:1789760772.246640:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.246689:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22040576, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.247448:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.247463:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.247467:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.247473:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.247486:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.288295:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:0.0:1789760772.288407:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22040576 -> 22044672 00010000:00010000:0.0:1789760772.288422:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288434:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288443:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288449:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.288454:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288462:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288480:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.288488:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.288586:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41945us (42247us total) trans 4294972701 rc 0/0 00000100:00100000:0.0:1789760772.288596:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5411 00000100:00100000:0.0:1789760772.288606:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753531644505 time 753561921111 state 0x2 ref 0 00000100:00100000:0.0:1789760772.298196:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.298352:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323520 00000100:00100000:2.0:1789760772.298363:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323520 00000100:00100000:2.0:1789760772.298418:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753531644505 time 753561921111 state 0x0 ref 1 00000100:00100000:2.0:1789760772.298435:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753562181527 time 753561921111 state 0x1 ref 1 00000100:00100000:2.0:1789760772.298469:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753562181527 time 753613771675 state 0x0 ref 1 00000100:00100000:2.0:1789760772.298482:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.298531:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22044672, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.298540:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.298556:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.298571:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.298578:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.298592:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.341629:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00002000:00010000:0.0:1789760772.341785:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:0.0:1789760772.341801:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341814:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341825:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341832:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.341836:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341845:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341863:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.341870:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b8f65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.341976:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507323520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43491us (43784us total) trans 4294972702 rc 0/0 00000100:00100000:0.0:1789760772.341990:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5412 00000100:00100000:0.0:1789760772.341998:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753562181527 time 753613771675 state 0x2 ref 0 00000100:00100000:2.0:1789760772.345412:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760772.345502:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323648 00000100:00100000:3.0:1789760772.345512:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323648 00000100:00100000:3.0:1789760772.345577:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753562181527 time 753613771675 state 0x0 ref 1 00000100:00100000:3.0:1789760772.345607:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753614032091 time 753613771675 state 0x1 ref 1 00000100:00100000:3.0:1789760772.345636:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753614032091 time 753660939451 state 0x0 ref 1 00000100:00100000:3.0:1789760772.345648:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507323648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760772.345870:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22048768, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:3.0:1789760772.345880:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.345896:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760772.345909:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.345917:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.345931:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.391619:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:0.0:1789760772.391760:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:0.0:1789760772.391776:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391790:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391799:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391807:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.391811:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391820:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391841:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.391849:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b8f6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.392209:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507323648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46557us (46801us total) trans 4294972703 rc 0/0 00000100:00100000:0.0:1789760772.392221:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5413 00000100:00100000:0.0:1789760772.392231:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753614032091 time 753660939451 state 0x2 ref 0 00000100:00100000:3.0:1789760772.406891:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.410923:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323776 00000100:00100000:0.0:1789760772.410939:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323776 00000100:00100000:2.0:1789760772.411335:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753614032091 time 753660939451 state 0x0 ref 1 00000100:00100000:2.0:1789760772.411351:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753661199867 time 753660939451 state 0x1 ref 1 00000100:00100000:2.0:1789760772.411384:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753661199867 time 753726686768 state 0x0 ref 1 00000100:00100000:2.0:1789760772.411398:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507323776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.411446:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22052864, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.411455:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.411470:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760772.411475:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.411481:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.411495:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.451853:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00002000:00010000:0.0:1789760772.451970:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:0.0:1789760772.451988:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452191:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452203:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452211:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.452215:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452224:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452245:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.452258:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b8f73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.452466:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507323776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41065us (45579us total) trans 4294972704 rc 0/0 00000100:00100000:0.0:1789760772.452477:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5414 00000100:00100000:0.0:1789760772.452486:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753661199867 time 753726686768 state 0x2 ref 0 00000100:00100000:3.0:1789760772.460159:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.460316:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507323904 00000100:00100000:0.0:1789760772.460326:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507323904 00000100:00100000:0.0:1789760772.460383:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753661199867 time 753726686768 state 0x0 ref 1 00000100:00100000:0.0:1789760772.460399:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753726947184 time 753726686768 state 0x1 ref 1 00000100:00100000:0.0:1789760772.460433:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753726947184 time 753775736326 state 0x0 ref 1 00000100:00100000:0.0:1789760772.460446:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507323904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.460492:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22056960, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.460501:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.460517:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.460521:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.460527:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.460545:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.502333:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:0.0:1789760772.502353:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502365:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502374:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502381:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.502386:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502395:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502415:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.502423:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.502529:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507323904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42076us (42371us total) trans 4294972705 rc 0/0 00000100:00100000:0.0:1789760772.502540:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5415 00000100:00100000:0.0:1789760772.502550:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753726947184 time 753775736326 state 0x2 ref 0 00000001:00080000:1.0:1789760772.502903:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00000100:00100000:1.0:1789760772.506505:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.506669:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324032 00000100:00100000:0.0:1789760772.506680:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324032 00000100:00100000:0.0:1789760772.506786:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753726947184 time 753775736326 state 0x0 ref 1 00000100:00100000:0.0:1789760772.506802:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753775996742 time 753775736326 state 0x1 ref 1 00000100:00100000:0.0:1789760772.506882:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753775996742 time 753822183305 state 0x0 ref 1 00000100:00100000:0.0:1789760772.506895:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507324032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.506941:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22061056, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.506950:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.506967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.506971:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.506977:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.506991:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.565125:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00002000:00010000:0.0:1789760772.565347:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:0.0:1789760772.565363:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565376:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565385:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565392:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.565396:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565404:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565428:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.565436:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.565529:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507324032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58632us (59029us total) trans 4294972706 rc 0/0 00000100:00100000:0.0:1789760772.565544:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5416 00000100:00100000:0.0:1789760772.565566:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753775996742 time 753822183305 state 0x2 ref 0 00000100:00100000:3.0:1789760772.573658:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.573809:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324160 00000100:00100000:0.0:1789760772.573820:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324160 00000100:00100000:0.0:1789760772.573880:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753775996742 time 753822183305 state 0x0 ref 1 00000100:00100000:0.0:1789760772.573898:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753822443721 time 753822183305 state 0x1 ref 1 00000100:00100000:0.0:1789760772.573931:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753822443721 time 753889234146 state 0x0 ref 1 00000100:00100000:0.0:1789760772.573945:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507324160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.574006:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22065152, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.574016:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.574032:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.574037:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.574044:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.574058:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.633962:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00002000:00010000:0.0:1789760772.634070:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22065152 -> 22069248 00010000:00010000:0.0:1789760772.634085:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634099:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634108:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634115:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.634120:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634129:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634149:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634157:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b8f88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.634248:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507324160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60301us (60594us total) trans 4294972707 rc 0/0 00000100:00100000:0.0:1789760772.634258:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5417 00000100:00100000:0.0:1789760772.634267:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753822443721 time 753889234146 state 0x2 ref 0 00000100:00100000:3.0:1789760772.639957:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.640071:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324288 00000100:00100000:0.0:1789760772.640081:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324288 00000100:00100000:0.0:1789760772.640138:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753822443721 time 753889234146 state 0x0 ref 1 00000100:00100000:0.0:1789760772.640153:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753889494562 time 753889234146 state 0x1 ref 1 00000100:00100000:0.0:1789760772.640186:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753889494562 time 753955488575 state 0x0 ref 1 00000100:00100000:0.0:1789760772.640198:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507324288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.640249:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22069248, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.640258:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.640274:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.640279:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.640286:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.640299:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.665716:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22069248 -> 22073344 00010000:00010000:1.0:1789760772.665735:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665748:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665758:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665765:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760772.665771:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665780:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665801:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665809:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b8f8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.665894:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507324288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25693us (25943us total) trans 4294972708 rc 0/0 00000100:00100000:1.0:1789760772.665904:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5418 00000100:00100000:1.0:1789760772.665914:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753889494562 time 753955488575 state 0x2 ref 0 00000001:00080000:3.0:1789760772.667156:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00000100:00100000:1.0:1789760772.669070:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.669202:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324416 00000100:00100000:0.0:1789760772.669212:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324416 00000100:00100000:0.0:1789760772.669267:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753889494562 time 753955488575 state 0x0 ref 1 00000100:00100000:0.0:1789760772.669282:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753955748991 time 753955488575 state 0x1 ref 1 00000100:00100000:0.0:1789760772.669314:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753955748991 time 753984616576 state 0x0 ref 1 00000100:00100000:0.0:1789760772.669329:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507324416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.669376:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22073344, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.669385:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.669401:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760772.669406:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.669412:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.669425:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.713829:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:0.0:1789760772.713851:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713863:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713873:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713880:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760772.713885:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713894:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713915:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713922:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8f96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.714027:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507324416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44695us (44960us total) trans 4294972709 rc 0/0 00000100:00100000:0.0:1789760772.714037:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5419 00000100:00100000:0.0:1789760772.714046:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753955748991 time 753984616576 state 0x2 ref 0 00000001:00080000:3.0:1789760772.714435:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00000100:00100000:0.0:1789760772.721940:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.722135:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324544 00000100:00100000:2.0:1789760772.722145:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324544 00000100:00100000:2.0:1789760772.722203:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753955748991 time 753984616576 state 0x0 ref 1 00000100:00100000:2.0:1789760772.722223:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753984876992 time 753984616576 state 0x1 ref 1 00000100:00100000:2.0:1789760772.722252:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753984876992 time 754037555574 state 0x0 ref 1 00000100:00100000:2.0:1789760772.722264:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507324544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760772.722315:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22077440, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:2.0:1789760772.722325:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.722349:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760772.722353:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.722360:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.722377:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.783855:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:1.0:1789760772.784291:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22077440 -> 22081536 00010000:00010000:1.0:1789760772.784306:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784318:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784328:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784336:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760772.784340:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784349:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784370:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784377:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b8f9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.784472:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507324544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62204us (62537us total) trans 4294972710 rc 0/0 00000100:00100000:1.0:1789760772.784485:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5420 00000100:00100000:1.0:1789760772.784495:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753984876992 time 754037555574 state 0x2 ref 0 00000100:00100000:1.0:1789760772.792765:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.792917:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324672 00000100:00100000:0.0:1789760772.792927:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324672 00000100:00100000:0.0:1789760772.792991:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 753984876992 time 754037555574 state 0x0 ref 1 00000100:00100000:0.0:1789760772.793017:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754037815990 time 754037555574 state 0x1 ref 1 00000100:00100000:0.0:1789760772.793046:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754037815990 time 754108349936 state 0x0 ref 1 00000100:00100000:0.0:1789760772.793068:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507324672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.793120:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22081536, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.793129:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.793144:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760772.793149:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.793154:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.793168:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.839762:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00002000:00010000:3.0:1789760772.840200:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:3.0:1789760772.840288:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840303:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840311:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840317:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760772.840322:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840331:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840351:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.840358:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b8fa4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.840453:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507324672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47392us (47693us total) trans 4294972711 rc 0/0 00000100:00100000:3.0:1789760772.840462:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5421 00000100:00100000:3.0:1789760772.840473:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754037815990 time 754108349936 state 0x2 ref 0 00000100:00100000:2.0:1789760772.852250:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760772.853016:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324800 00000100:00100000:0.0:1789760772.853027:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324800 00000100:00100000:0.0:1789760772.853086:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754037815990 time 754108349936 state 0x0 ref 1 00000100:00100000:0.0:1789760772.853102:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754108610352 time 754108349936 state 0x1 ref 1 00000100:00100000:0.0:1789760772.853249:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754108610352 time 754168551369 state 0x0 ref 1 00000100:00100000:0.0:1789760772.853263:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507324800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760772.853360:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22085632, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:0.0:1789760772.853370:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.853386:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760772.853391:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.853397:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.853411:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.913193:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00002000:00010000:1.0:1789760772.913429:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22085632 -> 22089728 00010000:00010000:1.0:1789760772.913444:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913456:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913465:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913471:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760772.913476:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913483:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913501:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913508:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b8fab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.913935:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507324800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60670us (61691us total) trans 4294972712 rc 0/0 00000100:00100000:1.0:1789760772.913946:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5422 00000100:00100000:1.0:1789760772.913955:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754108610352 time 754168551369 state 0x2 ref 0 00000100:00100000:0.0:1789760772.917336:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760772.917419:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507324928 00000100:00100000:1.0:1789760772.917429:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507324928 00000100:00100000:1.0:1789760772.917485:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754108610352 time 754168551369 state 0x0 ref 1 00000100:00100000:1.0:1789760772.917501:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754168811785 time 754168551369 state 0x1 ref 1 00000100:00100000:1.0:1789760772.917580:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754168811785 time 754232882069 state 0x0 ref 1 00000100:00100000:1.0:1789760772.917605:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507324928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760772.917651:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22089728, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:1.0:1789760772.917660:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.917676:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760772.917681:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.917687:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.917733:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.945890:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00002000:00010000:1.0:1789760772.945999:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:1.0:1789760772.946014:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946026:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946033:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946040:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760772.946045:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946054:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946072:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.946080:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.946199:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507324928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28603us (28868us total) trans 4294972713 rc 0/0 00000100:00100000:1.0:1789760772.946213:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5423 00000100:00100000:1.0:1789760772.946223:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754168811785 time 754232882069 state 0x2 ref 0 00000100:00100000:3.0:1789760772.953551:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760772.955179:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325056 00000100:00100000:2.0:1789760772.955192:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325056 00000100:00100000:3.0:1789760772.955314:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754168811785 time 754232882069 state 0x0 ref 1 00000100:00100000:3.0:1789760772.955331:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754233142485 time 754232882069 state 0x1 ref 1 00000100:00100000:3.0:1789760772.955393:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754233142485 time 754270695875 state 0x0 ref 1 00000100:00100000:3.0:1789760772.955407:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760772.955456:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22093824, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:3.0:1789760772.955465:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.955481:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760772.955487:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.955493:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.955507:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.994116:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00002000:00010000:1.0:1789760772.994262:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:1.0:1789760772.994278:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994291:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994300:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994307:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760772.994311:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994321:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994341:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.994349:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b8fb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.994475:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39067us (40931us total) trans 4294972714 rc 0/0 00000100:00100000:1.0:1789760772.994485:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5424 00000100:00100000:1.0:1789760772.994494:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754233142485 time 754270695875 state 0x2 ref 0 00000100:00100000:3.0:1789760773.002395:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.002504:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325184 00000100:00100000:1.0:1789760773.002515:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325184 00000100:00100000:1.0:1789760773.002638:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754233142485 time 754270695875 state 0x0 ref 1 00000100:00100000:1.0:1789760773.002652:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754270956291 time 754270695875 state 0x1 ref 1 00000100:00100000:1.0:1789760773.002680:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754270956291 time 754317983516 state 0x0 ref 1 00000100:00100000:1.0:1789760773.002714:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507325184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.002759:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22097920, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb007 00010000:00010000:1.0:1789760773.002768:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.002783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.002788:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.002794:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.002807:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760773.003520:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760773.020271:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760773.033884:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22097920 -> 22102016 00002000:00010000:1.0:1789760773.033892:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:1.0:1789760773.033895:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:1.0:1789760773.033908:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033931:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033941:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033948:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.033953:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033962:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033982:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.033989:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.034097:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507325184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31401us (31707us total) trans 4294972715 rc 0/0 00000100:00100000:1.0:1789760773.034107:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5425 00000100:00100000:1.0:1789760773.034116:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754270956291 time 754317983516 state 0x2 ref 0 00000001:00080000:0.0:1789760773.035208:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00000100:00100000:0.0:1789760773.044374:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.044432:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325312 00000100:00100000:1.0:1789760773.044442:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325312 00000100:00100000:1.0:1789760773.044495:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754270956291 time 754317983516 state 0x0 ref 1 00000100:00100000:1.0:1789760773.044511:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754318243932 time 754317983516 state 0x1 ref 1 00000100:00100000:1.0:1789760773.044540:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754318243932 time 754359843714 state 0x0 ref 1 00000100:00100000:1.0:1789760773.044565:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.044611:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22102016, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.044619:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.044635:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.044639:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.044645:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.044663:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.097891:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:1.0:1789760773.098070:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22102016 -> 22106112 00010000:00010000:1.0:1789760773.098085:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098098:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098108:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098116:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.098120:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098130:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098149:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.098157:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.098248:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53691us (53876us total) trans 4294972716 rc 0/0 00000100:00100000:1.0:1789760773.098261:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5426 00000100:00100000:1.0:1789760773.098271:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754318243932 time 754359843714 state 0x2 ref 0 00000100:00100000:3.0:1789760773.108914:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.109001:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325440 00000100:00100000:1.0:1789760773.109012:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325440 00000100:00100000:1.0:1789760773.109067:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754318243932 time 754359843714 state 0x0 ref 1 00000100:00100000:1.0:1789760773.109083:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754360104130 time 754359843714 state 0x1 ref 1 00000100:00100000:1.0:1789760773.109112:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754360104130 time 754424414696 state 0x0 ref 1 00000100:00100000:1.0:1789760773.109123:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507325440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.109169:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22106112, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.109179:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.109194:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.109198:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.109204:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.109218:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.162424:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00002000:00010000:1.0:1789760773.162600:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:1.0:1789760773.162616:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162629:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162638:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162646:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.162650:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162660:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162811:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.162821:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.162955:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507325440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53827us (54045us total) trans 4294972717 rc 0/0 00000100:00100000:1.0:1789760773.162966:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5427 00000100:00100000:1.0:1789760773.162975:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754360104130 time 754424414696 state 0x2 ref 0 00000100:00100000:3.0:1789760773.165820:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.165924:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325568 00000100:00100000:1.0:1789760773.165934:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325568 00000100:00100000:1.0:1789760773.165987:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754360104130 time 754424414696 state 0x0 ref 1 00000100:00100000:1.0:1789760773.166001:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754424675112 time 754424414696 state 0x1 ref 1 00000100:00100000:1.0:1789760773.166041:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754424675112 time 754481335499 state 0x0 ref 1 00000100:00100000:1.0:1789760773.166054:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.166106:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22110208, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.166115:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.166130:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.166134:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.166140:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.166154:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.206887:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00002000:00010000:1.0:1789760773.207162:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:1.0:1789760773.207178:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207191:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207207:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207214:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.207219:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207229:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207248:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.207255:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b8fd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.207347:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41290us (41532us total) trans 4294972718 rc 0/0 00000100:00100000:1.0:1789760773.207357:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5428 00000100:00100000:1.0:1789760773.207367:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754424675112 time 754481335499 state 0x2 ref 0 00000100:00100000:0.0:1789760773.211212:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.211288:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325696 00000100:00100000:1.0:1789760773.211299:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325696 00000100:00100000:1.0:1789760773.211352:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754424675112 time 754481335499 state 0x0 ref 1 00000100:00100000:1.0:1789760773.211367:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754481595915 time 754481335499 state 0x1 ref 1 00000100:00100000:1.0:1789760773.211396:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754481595915 time 754526700071 state 0x0 ref 1 00000100:00100000:1.0:1789760773.211407:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507325696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.211456:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22114304, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.211467:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.211482:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.211487:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.211494:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.211508:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.250455:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00002000:00010000:3.0:1789760773.250577:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:3.0:1789760773.250592:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250605:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250614:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250623:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760773.250628:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250636:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250661:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.250669:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.250991:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507325696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39579us (39782us total) trans 4294972719 rc 0/0 00000100:00100000:3.0:1789760773.251002:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5429 00000100:00100000:3.0:1789760773.251011:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754481595915 time 754526700071 state 0x2 ref 0 00000100:00100000:2.0:1789760773.257274:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.257337:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325824 00000100:00100000:1.0:1789760773.257346:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325824 00000100:00100000:1.0:1789760773.257480:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754481595915 time 754526700071 state 0x0 ref 1 00000100:00100000:1.0:1789760773.257497:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754526960487 time 754526700071 state 0x1 ref 1 00000100:00100000:1.0:1789760773.257529:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754526960487 time 754572832351 state 0x0 ref 1 00000100:00100000:1.0:1789760773.257543:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.257605:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22118400, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.257614:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.257630:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.257634:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.257641:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.257655:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.305963:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00002000:00010000:1.0:1789760773.306137:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:1.0:1789760773.306153:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306167:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306176:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306184:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.306188:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306198:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306219:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.306226:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8fe3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.306347:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507325824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48802us (49078us total) trans 4294972720 rc 0/0 00000100:00100000:1.0:1789760773.306359:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5430 00000100:00100000:1.0:1789760773.306369:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754526960487 time 754572832351 state 0x2 ref 0 00000100:00100000:1.0:1789760773.313217:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.313370:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507325952 00000100:00100000:2.0:1789760773.313380:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507325952 00000100:00100000:2.0:1789760773.313440:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754526960487 time 754572832351 state 0x0 ref 1 00000100:00100000:2.0:1789760773.314064:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754573092767 time 754572832351 state 0x1 ref 1 00000100:00100000:2.0:1789760773.314142:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754573092767 time 754629444768 state 0x0 ref 1 00000100:00100000:2.0:1789760773.314158:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507325952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760773.314209:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22122496, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:2.0:1789760773.314218:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.314232:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760773.314237:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.314242:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.314255:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.353030:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00002000:00010000:1.0:1789760773.353190:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:1.0:1789760773.353205:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353219:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353229:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353236:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.353242:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353256:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353277:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.353286:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b8fea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.353520:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507325952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39361us (40307us total) trans 4294972721 rc 0/0 00000100:00100000:1.0:1789760773.353531:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5431 00000100:00100000:1.0:1789760773.353541:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754573092767 time 754629444768 state 0x2 ref 0 00000100:00100000:0.0:1789760773.357365:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760773.357495:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326080 00000100:00100000:3.0:1789760773.357506:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326080 00000100:00100000:3.0:1789760773.357565:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754573092767 time 754629444768 state 0x0 ref 1 00000100:00100000:3.0:1789760773.357581:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754629705184 time 754629444768 state 0x1 ref 1 00000100:00100000:3.0:1789760773.357625:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754629705184 time 754672927830 state 0x0 ref 1 00000100:00100000:3.0:1789760773.357637:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760773.357686:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22126592, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:3.0:1789760773.357720:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.357741:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760773.357746:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.357753:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.357766:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.381916:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00002000:00010000:1.0:1789760773.382019:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22126592 -> 22130688 00010000:00010000:1.0:1789760773.382034:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382046:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382054:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382061:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.382066:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382074:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382091:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.382099:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b8ff1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.382186:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24546us (24836us total) trans 4294972722 rc 0/0 00000100:00100000:1.0:1789760773.382196:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5432 00000100:00100000:1.0:1789760773.382205:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754629705184 time 754672927830 state 0x2 ref 0 00000100:00100000:3.0:1789760773.388642:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.388886:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326208 00000100:00100000:1.0:1789760773.388903:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326208 00000100:00100000:1.0:1789760773.388959:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754629705184 time 754672927830 state 0x0 ref 1 00000100:00100000:1.0:1789760773.388974:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754673188246 time 754672927830 state 0x1 ref 1 00000100:00100000:1.0:1789760773.389005:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754673188246 time 754704308780 state 0x0 ref 1 00000100:00100000:1.0:1789760773.389016:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.389063:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22130688, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.389072:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.389088:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.389092:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.389098:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.389111:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.413414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00002000:00010000:3.0:1789760773.413511:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:3.0:1789760773.413526:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413537:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413547:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413563:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760773.413568:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413577:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413666:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.413674:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b8ff8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.413886:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24866us (25249us total) trans 4294972723 rc 0/0 00000100:00100000:3.0:1789760773.413897:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5433 00000100:00100000:3.0:1789760773.413906:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754673188246 time 754704308780 state 0x2 ref 0 00000100:00100000:3.0:1789760773.417643:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.417986:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326336 00000100:00100000:1.0:1789760773.418006:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326336 00000100:00100000:1.0:1789760773.418061:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754673188246 time 754704308780 state 0x0 ref 1 00000100:00100000:1.0:1789760773.418076:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754704569196 time 754704308780 state 0x1 ref 1 00000100:00100000:1.0:1789760773.418108:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754704569196 time 754733411535 state 0x0 ref 1 00000100:00100000:1.0:1789760773.418121:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.418169:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22134784, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.418178:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.418194:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.418199:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.418205:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.418220:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.442827:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22134784 -> 22138880 00010000:00010000:2.0:1789760773.442847:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442861:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442871:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442885:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.442890:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442900:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442922:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.442930:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b8fff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.443028:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507326336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24905us (25515us total) trans 4294972724 rc 0/0 00000100:00100000:2.0:1789760773.443039:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5434 00000100:00100000:2.0:1789760773.443048:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754704569196 time 754733411535 state 0x2 ref 0 00000001:00080000:2.0:1789760773.443084:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00000100:00100000:2.0:1789760773.447019:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.447482:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326464 00000100:00100000:1.0:1789760773.447493:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326464 00000100:00100000:1.0:1789760773.447556:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754704569196 time 754733411535 state 0x0 ref 1 00000100:00100000:1.0:1789760773.447573:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754733671951 time 754733411535 state 0x1 ref 1 00000100:00100000:1.0:1789760773.447605:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754733671951 time 754762908317 state 0x0 ref 1 00000100:00100000:1.0:1789760773.447619:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.447664:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22138880, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.447673:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447690:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.447725:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.447731:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.448111:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.472293:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00002000:00010000:2.0:1789760773.472497:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:2.0:1789760773.472512:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472524:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472532:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472539:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.472544:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472567:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472650:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.472660:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9006 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.472874:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25253us (25859us total) trans 4294972725 rc 0/0 00000100:00100000:2.0:1789760773.472885:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5435 00000100:00100000:2.0:1789760773.472894:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754733671951 time 754762908317 state 0x2 ref 0 00000100:00100000:3.0:1789760773.476075:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.476161:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326592 00000100:00100000:1.0:1789760773.476172:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326592 00000100:00100000:1.0:1789760773.476226:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754733671951 time 754762908317 state 0x0 ref 1 00000100:00100000:1.0:1789760773.476241:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754763168733 time 754762908317 state 0x1 ref 1 00000100:00100000:1.0:1789760773.476273:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754763168733 time 754791577462 state 0x0 ref 1 00000100:00100000:1.0:1789760773.476286:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.476338:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22142976, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.476348:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.476364:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.476369:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.476375:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.476388:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.501678:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00002000:00010000:1.0:1789760773.501809:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:1.0:1789760773.501825:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501837:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501851:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501858:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.501863:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501873:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501895:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.501902:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b900d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.502427:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25711us (25930us total) trans 4294972726 rc 0/0 00000100:00100000:1.0:1789760773.502439:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5436 00000100:00100000:1.0:1789760773.502448:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754763168733 time 754791577462 state 0x2 ref 0 00000100:00100000:3.0:1789760773.507909:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.508082:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326720 00000100:00100000:2.0:1789760773.508092:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326720 00000100:00100000:2.0:1789760773.508148:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754763168733 time 754791577462 state 0x0 ref 1 00000100:00100000:2.0:1789760773.508171:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754791837878 time 754791577462 state 0x1 ref 1 00000100:00100000:2.0:1789760773.508198:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754791837878 time 754823501873 state 0x0 ref 1 00000100:00100000:2.0:1789760773.508212:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760773.508260:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22147072, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:2.0:1789760773.508269:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.508285:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760773.508290:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.508297:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.508313:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.532067:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00002000:00010000:2.0:1789760773.532397:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:2.0:1789760773.532412:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532424:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532434:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532443:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.532448:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532458:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532478:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.532486:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9014 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.532579:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24362us (24673us total) trans 4294972727 rc 0/0 00000100:00100000:2.0:1789760773.532598:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5437 00000100:00100000:2.0:1789760773.532607:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754791837878 time 754823501873 state 0x2 ref 0 00000100:00100000:2.0:1789760773.536202:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760773.536298:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326848 00000100:00100000:0.0:1789760773.536308:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326848 00000100:00100000:0.0:1789760773.536363:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754791837878 time 754823501873 state 0x0 ref 1 00000100:00100000:0.0:1789760773.536378:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754823762289 time 754823501873 state 0x1 ref 1 00000100:00100000:0.0:1789760773.536436:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754823762289 time 754851738854 state 0x0 ref 1 00000100:00100000:0.0:1789760773.536448:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760773.536494:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22151168, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:0.0:1789760773.536503:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.536519:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760773.536523:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.536530:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.536543:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.557573:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00002000:00010000:0.0:1789760773.557916:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:0.0:1789760773.557931:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.557943:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.557951:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.557959:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760773.557964:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.557974:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.557993:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.558002:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b901b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.558127:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507326848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21675us (21931us total) trans 4294972728 rc 0/0 00000100:00100000:0.0:1789760773.558139:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5438 00000100:00100000:0.0:1789760773.558149:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754823762289 time 754851738854 state 0x2 ref 0 00000100:00100000:3.0:1789760773.561803:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.561932:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507326976 00000100:00100000:1.0:1789760773.561942:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507326976 00000100:00100000:1.0:1789760773.562005:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754823762289 time 754851738854 state 0x0 ref 1 00000100:00100000:1.0:1789760773.562022:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754851999270 time 754851738854 state 0x1 ref 1 00000100:00100000:1.0:1789760773.562050:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754851999270 time 754877353489 state 0x0 ref 1 00000100:00100000:1.0:1789760773.562062:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.562105:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22155264, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.562114:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.562128:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760773.562131:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.562137:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.562151:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.618970:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00002000:00010000:2.0:1789760773.619515:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:2.0:1789760773.619531:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619543:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619563:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619571:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.619576:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619690:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619778:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.619786:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9022 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.619916:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507326976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57851us (58120us total) trans 4294972729 rc 0/0 00000100:00100000:2.0:1789760773.619926:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5439 00000100:00100000:2.0:1789760773.619935:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754851999270 time 754877353489 state 0x2 ref 0 00000100:00100000:3.0:1789760773.625079:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.625145:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327104 00000100:00100000:2.0:1789760773.625154:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327104 00000100:00100000:2.0:1789760773.625201:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754851999270 time 754877353489 state 0x0 ref 1 00000100:00100000:2.0:1789760773.625216:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754877613905 time 754877353489 state 0x1 ref 1 00000100:00100000:2.0:1789760773.625245:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754877613905 time 754940548276 state 0x0 ref 1 00000100:00100000:2.0:1789760773.625256:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507327104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760773.625299:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22159360, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:2.0:1789760773.625307:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.625322:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760773.625327:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.625333:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.625346:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.657515:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00002000:00010000:2.0:1789760773.657681:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:2.0:1789760773.657722:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657737:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657746:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657753:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.657758:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657767:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657786:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.657794:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9029 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.657929:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507327104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32669us (32854us total) trans 4294972730 rc 0/0 00000100:00100000:2.0:1789760773.657940:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5440 00000100:00100000:2.0:1789760773.657949:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754877613905 time 754940548276 state 0x2 ref 0 00000100:00100000:0.0:1789760773.664800:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.665889:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327232 00000100:00100000:2.0:1789760773.665901:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327232 00000100:00100000:2.0:1789760773.665959:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754877613905 time 754940548276 state 0x0 ref 1 00000100:00100000:2.0:1789760773.665974:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754940808692 time 754940548276 state 0x1 ref 1 00000100:00100000:2.0:1789760773.666108:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754940808692 time 754981410025 state 0x0 ref 1 00000100:00100000:2.0:1789760773.666122:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507327232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760773.666169:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22163456, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:2.0:1789760773.666179:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.666195:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760773.666200:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.666207:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.666221:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.689409:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00002000:00010000:2.0:1789760773.689497:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22163456 -> 22167552 00010000:00010000:2.0:1789760773.689518:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689531:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689541:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689673:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.689679:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689689:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689786:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.689795:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9030 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.689882:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507327232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23758us (25088us total) trans 4294972731 rc 0/0 00000100:00100000:2.0:1789760773.689893:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5441 00000100:00100000:2.0:1789760773.689901:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754940808692 time 754981410025 state 0x2 ref 0 00000100:00100000:2.0:1789760773.693184:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.693297:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327360 00000100:00100000:1.0:1789760773.693304:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327360 00000100:00100000:1.0:1789760773.693347:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754940808692 time 754981410025 state 0x0 ref 1 00000100:00100000:1.0:1789760773.693359:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754981670441 time 754981410025 state 0x1 ref 1 00000100:00100000:1.0:1789760773.693402:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754981670441 time 755008706442 state 0x0 ref 1 00000100:00100000:1.0:1789760773.693411:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507327360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.693451:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22167552, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.693468:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.693482:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760773.693487:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.693493:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.693508:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.731993:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22167552 -> 22171648 00010000:00010000:1.0:1789760773.732018:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732035:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732047:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732056:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.732062:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732073:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732098:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.732107:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9037 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.732227:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507327360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38806us (39040us total) trans 4294972732 rc 0/0 00000100:00100000:1.0:1789760773.732239:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5442 00000100:00100000:1.0:1789760773.732252:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754981670441 time 755008706442 state 0x2 ref 0 00000001:00080000:3.0:1789760773.734745:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00000100:00100000:3.0:1789760773.736986:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760773.737097:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327488 00000100:00100000:0.0:1789760773.737111:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327488 00000100:00100000:0.0:1789760773.737186:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 754981670441 time 755008706442 state 0x0 ref 1 00000100:00100000:0.0:1789760773.737205:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755008966858 time 755008706442 state 0x1 ref 1 00000100:00100000:0.0:1789760773.737238:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755008966858 time 755052541308 state 0x0 ref 1 00000100:00100000:0.0:1789760773.737253:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507327488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760773.737320:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22171648, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:0.0:1789760773.737330:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737361:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760773.737366:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.737537:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737566:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.782895:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:2.0:1789760773.782922:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.782942:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.782953:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.782961:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.782966:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.782983:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.783064:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00010000:00010000:2.0:1789760773.783119:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.783127:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b903e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.783229:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507327488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45972us (46249us total) trans 4294972733 rc 0/0 00000100:00100000:2.0:1789760773.783241:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5443 00000100:00100000:2.0:1789760773.783251:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755008966858 time 755052541308 state 0x2 ref 0 00000100:00100000:3.0:1789760773.792361:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760773.792831:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327616 00000100:00100000:3.0:1789760773.792846:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327616 00000100:00100000:3.0:1789760773.792924:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755008966858 time 755052541308 state 0x0 ref 1 00000100:00100000:3.0:1789760773.792944:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755052801724 time 755052541308 state 0x1 ref 1 00000100:00100000:3.0:1789760773.792979:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755052801724 time 755108281946 state 0x0 ref 1 00000100:00100000:3.0:1789760773.793093:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507327616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760773.793160:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22175744, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:3.0:1789760773.793171:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.793191:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760773.793196:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.793204:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.793222:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.822355:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:1.0:1789760773.822612:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22175744 -> 22179840 00010000:00010000:1.0:1789760773.822633:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822648:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822659:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822667:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760773.822673:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822690:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822749:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.822810:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9045 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.822933:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507327616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29935us (30581us total) trans 4294972734 rc 0/0 00000100:00100000:1.0:1789760773.822945:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5444 00000100:00100000:1.0:1789760773.822957:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755052801724 time 755108281946 state 0x2 ref 0 00000100:00100000:0.0:1789760773.826974:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.827155:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327744 00000100:00100000:1.0:1789760773.827169:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327744 00000100:00100000:1.0:1789760773.827243:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755052801724 time 755108281946 state 0x0 ref 1 00000100:00100000:1.0:1789760773.827261:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755108542362 time 755108281946 state 0x1 ref 1 00000100:00100000:1.0:1789760773.827349:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755108542362 time 755142651140 state 0x0 ref 1 00000100:00100000:1.0:1789760773.827365:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507327744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.827433:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22179840, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.827445:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.827466:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760773.827471:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.827478:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.827495:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.846309:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.846838:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328000 00000100:00100000:2.0:1789760773.846852:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328000 00000100:00080000:1.0:1789760773.847137:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760773.847270:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 948 00000100:00100000:2.0:1789760773.847283:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507328000:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760773.847391:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507328000:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 111us (1096us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.847400:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 948 00000100:00100000:2.0:1789760773.851233:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.851278:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.851310:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.851334:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760773.851366:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328128 00000100:00100000:2.0:1789760773.851378:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328128 00000100:00100000:2.0:1789760773.851446:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328384 00000100:00100000:2.0:1789760773.851450:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328384 00000100:00100000:2.0:1789760773.851468:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328256 00000100:00100000:2.0:1789760773.851470:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328256 00000100:00100000:2.0:1789760773.851492:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 949 00000100:00100000:2.0:1789760773.851503:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507328128:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760773.851617:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507328128:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 116us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.851625:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 949 00000100:00100000:2.0:1789760773.851643:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 950 00000100:00100000:2.0:1789760773.851650:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507328384:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760773.851675:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2416:x1876699507328384:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 26us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.851681:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 950 00000100:00100000:2.0:1789760773.851738:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 951 00000100:00100000:2.0:1789760773.851767:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507328256:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760773.851798:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507328256:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 52us (463us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.851804:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 951 00000100:00080000:2.0:1789760773.851825:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760773.851830:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760773.853301:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507327872 00000100:00100000:3.0:1789760773.853314:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507327872 00000100:00100000:3.0:1789760773.853385:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 480 00000100:00100000:3.0:1789760773.853395:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507327872:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.853463:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2416:x1876699507327872:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 70us (2246us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.853470:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 480 00002000:00010000:2.0:1789760773.861949:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:2.0:1789760773.861976:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.862046:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00010000:00010000:2.0:1789760773.862117:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.862127:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.862135:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760773.862141:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.862150:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.862176:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.862184:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b904c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.862307:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507327744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34938us (35343us total) trans 4294972735 rc 0/0 00000100:00100000:2.0:1789760773.862320:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5445 00000100:00100000:2.0:1789760773.862333:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755108542362 time 755142651140 state 0x2 ref 0 00000100:00100000:2.0:1789760773.865107:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.865477:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328512 00000100:00100000:1.0:1789760773.865491:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328512 00000100:00100000:1.0:1789760773.865578:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755108542362 time 755142651140 state 0x0 ref 1 00000100:00100000:1.0:1789760773.865605:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755142911556 time 755142651140 state 0x1 ref 1 00000100:00100000:1.0:1789760773.865649:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755142911556 time 755180950666 state 0x0 ref 1 00000100:00100000:1.0:1789760773.865665:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507328512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.865771:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22183936, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.865782:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.865802:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760773.865808:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.865816:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.865838:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.908716:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00002000:00010000:3.0:1789760773.908810:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:3.0:1789760773.908827:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908841:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908851:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908858:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760773.908863:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908873:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908895:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.908902:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9053 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.909013:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507328512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43345us (43913us total) trans 4294972736 rc 0/0 00000100:00100000:3.0:1789760773.909026:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5446 00000100:00100000:3.0:1789760773.909037:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755142911556 time 755180950666 state 0x2 ref 0 00000100:00100000:0.0:1789760773.918891:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.918995:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328640 00000100:00100000:1.0:1789760773.919006:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328640 00000100:00100000:1.0:1789760773.919072:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755142911556 time 755180950666 state 0x0 ref 1 00000100:00100000:1.0:1789760773.919090:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755181211082 time 755180950666 state 0x1 ref 1 00000100:00100000:1.0:1789760773.919122:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755181211082 time 755234425635 state 0x0 ref 1 00000100:00100000:1.0:1789760773.919133:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507328640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.919196:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22188032, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.919206:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.919226:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760773.919231:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.919238:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.919255:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.974165:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00002000:00010000:3.0:1789760773.974476:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:3.0:1789760773.974499:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974514:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974524:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974533:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760773.974539:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974558:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974593:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.974603:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b905a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.975403:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507328640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56261us (56517us total) trans 4294972737 rc 0/0 00000100:00100000:3.0:1789760773.975420:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5447 00000100:00100000:3.0:1789760773.975432:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755181211082 time 755234425635 state 0x2 ref 0 00000100:00100000:2.0:1789760773.983599:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760773.983739:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328768 00000100:00100000:1.0:1789760773.983754:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328768 00000100:00100000:1.0:1789760773.983832:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755181211082 time 755234425635 state 0x0 ref 1 00000100:00100000:1.0:1789760773.983850:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755234686051 time 755234425635 state 0x1 ref 1 00000100:00100000:1.0:1789760773.983882:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755234686051 time 755299185143 state 0x0 ref 1 00000100:00100000:1.0:1789760773.983895:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507328768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760773.983962:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22192128, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:1.0:1789760773.983973:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983992:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760773.983997:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.984005:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.984023:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.045138:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760774.049849:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00002000:00010000:2.0:1789760774.050068:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:2.0:1789760774.050083:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050096:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050106:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050114:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.050120:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050130:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050150:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.050159:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9061 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.050257:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507328768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66358us (66673us total) trans 4294972738 rc 0/0 00000100:00100000:2.0:1789760774.050272:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5448 00000100:00100000:2.0:1789760774.050282:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755234686051 time 755299185143 state 0x2 ref 0 00000100:00100000:0.0:1789760774.055001:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760774.055078:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507328896 00000100:00100000:3.0:1789760774.055088:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507328896 00000100:00100000:3.0:1789760774.055140:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755234686051 time 755299185143 state 0x0 ref 1 00000100:00100000:3.0:1789760774.055156:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755299445559 time 755299185143 state 0x1 ref 1 00000100:00100000:3.0:1789760774.055217:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755299445559 time 755370518847 state 0x0 ref 1 00000100:00100000:3.0:1789760774.055231:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507328896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760774.055277:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22196224, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb007 00010000:00010000:3.0:1789760774.055286:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.055302:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760774.055307:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.055314:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.055330:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.150909:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00002000:00010000:0.0:1789760774.151030:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22196224 -> 22200320 00002000:00010000:0.0:1789760774.151034:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760774.151037:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:0.0:1789760774.151052:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151066:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151076:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151084:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760774.151089:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151100:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151125:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.151134:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9068 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.151262:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507328896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 96028us (96266us total) trans 4294972739 rc 0/0 00000100:00100000:0.0:1789760774.151273:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5449 00000100:00100000:0.0:1789760774.151292:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755299445559 time 755370518847 state 0x2 ref 0 00000100:00100000:1.0:1789760774.157361:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760774.158037:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329024 00000100:00100000:3.0:1789760774.158048:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329024 00000100:00100000:3.0:1789760774.158103:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755299445559 time 755370518847 state 0x0 ref 1 00000100:00100000:3.0:1789760774.158119:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755370779263 time 755370518847 state 0x1 ref 1 00000100:00100000:3.0:1789760774.158196:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755370779263 time 755473498713 state 0x0 ref 1 00000100:00100000:3.0:1789760774.158209:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507329024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760774.158258:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22200320, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:3.0:1789760774.158267:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.158293:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760774.158298:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.158304:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.158320:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.186941:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00002000:00010000:3.0:1789760774.187061:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:3.0:1789760774.187075:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187088:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187104:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187112:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.187117:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187127:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187147:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.187156:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b906f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.187249:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507329024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29036us (29894us total) trans 4294972740 rc 0/0 00000100:00100000:3.0:1789760774.187259:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5450 00000100:00100000:3.0:1789760774.187269:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755370779263 time 755473498713 state 0x2 ref 0 00000100:00100000:2.0:1789760774.196050:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760774.196513:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329152 00000100:00100000:1.0:1789760774.196523:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329152 00000100:00100000:1.0:1789760774.196672:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755370779263 time 755473498713 state 0x0 ref 1 00000100:00100000:1.0:1789760774.196689:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755473759129 time 755473498713 state 0x1 ref 1 00000100:00100000:1.0:1789760774.196737:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755473759129 time 755512040132 state 0x0 ref 1 00000100:00100000:1.0:1789760774.196752:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507329152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760774.196799:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22204416, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:1.0:1789760774.196808:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.196824:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760774.196829:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.196835:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.196849:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.232040:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00002000:00010000:1.0:1789760774.232126:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:1.0:1789760774.232141:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232154:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232162:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232169:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760774.232173:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232183:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232201:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232208:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9076 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.232292:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507329152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35538us (36247us total) trans 4294972741 rc 0/0 00000100:00100000:1.0:1789760774.232301:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5451 00000100:00100000:1.0:1789760774.232310:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755473759129 time 755512040132 state 0x2 ref 0 00000100:00100000:2.0:1789760774.240684:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760774.240918:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329280 00000100:00100000:0.0:1789760774.240929:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329280 00000100:00100000:0.0:1789760774.240992:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755473759129 time 755512040132 state 0x0 ref 1 00000100:00100000:0.0:1789760774.241006:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755512300548 time 755512040132 state 0x1 ref 1 00000100:00100000:0.0:1789760774.241033:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755512300548 time 755556336683 state 0x0 ref 1 00000100:00100000:0.0:1789760774.241045:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760774.241092:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22208512, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:0.0:1789760774.241100:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.241115:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760774.241121:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.241128:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.241141:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.284395:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00002000:00010000:2.0:1789760774.284553:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22208512 -> 22212608 00010000:00010000:2.0:1789760774.284569:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284713:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284724:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284779:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.284786:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284796:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284818:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.284826:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b907d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.284948:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43899us (44267us total) trans 4294972742 rc 0/0 00000100:00100000:2.0:1789760774.284959:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5452 00000100:00100000:2.0:1789760774.284968:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755512300548 time 755556336683 state 0x2 ref 0 00000100:00100000:2.0:1789760774.288798:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760774.288933:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329408 00000100:00100000:3.0:1789760774.288943:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329408 00000100:00100000:3.0:1789760774.289051:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755512300548 time 755556336683 state 0x0 ref 1 00000100:00100000:3.0:1789760774.289067:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755556597099 time 755556336683 state 0x1 ref 1 00000100:00100000:3.0:1789760774.289090:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755556597099 time 755604394162 state 0x0 ref 1 00000100:00100000:3.0:1789760774.289102:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507329408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760774.289148:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22212608, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:3.0:1789760774.289156:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.289172:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760774.289177:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.289183:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.289196:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.352980:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:2.0:1789760774.353075:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353090:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353099:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353108:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.353112:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353122:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353149:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.353158:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9084 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.353254:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507329408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64149us (64459us total) trans 4294972743 rc 0/0 00000100:00100000:2.0:1789760774.353265:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5453 00000100:00100000:2.0:1789760774.353274:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755556597099 time 755604394162 state 0x2 ref 0 00000001:00080000:0.0:1789760774.353509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00000100:00100000:0.0:1789760774.361297:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760774.364779:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23473b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538287872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.364850:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.364897:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538288384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.364925:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.364945:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538288896:0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.364975:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538288000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.364991:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760774.365014:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.365019:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538289408:0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365043:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538288512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365050:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760774.365077:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.365098:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287872 00000100:00100000:2.0:1789760774.365102:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538289024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365106:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287872 00000100:00100000:2.0:1789760774.365127:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760774.365147:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c10819ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538289536:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760774.365168:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:2.0:1789760774.365174:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.365181:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538287872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365208:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288000 00000100:00100000:2.0:1789760774.365214:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288000 00000100:00100000:3.0:1789760774.365237:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538287872:12345-0@lo:400:kworker.0 Request processed in 59us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.365247:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:2.0:1789760774.365248:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288512 00000100:00100000:2.0:1789760774.365253:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288512 00000100:00100000:2.0:1789760774.365283:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289024 00000100:00100000:2.0:1789760774.365286:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289024 00000100:00100000:2.0:1789760774.365308:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 952 00000100:00100000:2.0:1789760774.365315:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538288000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365353:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3462:x1876699538288000:12345-0@lo:400:kworker.0 Request processed in 37us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760774.365360:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 952 00000100:00100000:2.0:1789760774.365373:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 953 00000100:00100000:2.0:1789760774.365379:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538288512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365405:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3462:x1876699538288512:12345-0@lo:400:kworker.0 Request processed in 26us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760774.365412:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 953 00000100:00100000:3.0:1789760774.365419:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288896 00000100:00100000:3.0:1789760774.365425:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288896 00000100:00100000:2.0:1789760774.365429:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 954 00000100:00100000:2.0:1789760774.365437:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538289024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365463:0:14617:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289408 00000100:00100000:2.0:1789760774.365470:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3462:x1876699538289024:12345-0@lo:400:kworker.0 Request processed in 34us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760774.365476:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 954 00000100:00080000:2.0:1789760774.365504:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760774.365508:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760774.365539:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23462bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3462:1876699538288000:0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365561:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3462:1876699538288512:0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365570:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3462:1876699538289024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365588:0:14617:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289408 00000100:00100000:2.0:1789760774.365612:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289536 00000100:00100000:2.0:1789760774.365617:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289536 00000100:00100000:3.0:1789760774.365628:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:3.0:1789760774.365638:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538288896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365652:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789760774.365663:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538289536:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760774.365677:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538288896:12345-0@lo:400:kworker.0 Request processed in 41us (686us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.365684:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00002000:00000024:2.0:1789760774.365716:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760774.365720:0:14617:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:3.0:1789760774.365729:0:14617:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538289408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.365766:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538289536:12345-0@lo:13:osp-pre-1-1.0 Request processed in 105us (591us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.365773:0:14617:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538289408:12345-0@lo:400:kworker.0 Request processed in 42us (721us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760774.365778:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760774.365786:0:14617:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00080000:3.0:1789760774.365818:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760774.365831:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c10819ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538289536:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760774.365841:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23473b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538287872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365853:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538288896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.365862:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538289408:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367115:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234738e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538287744:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760774.367168:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.367191:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538288256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367216:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.367236:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538288768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367257:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.367276:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538289280:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367297:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.367318:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1789760774.367325:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538288128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367350:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.367373:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538287744 00000100:00100000:0.0:1789760774.367375:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538288640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.367381:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538287744 00000100:00100000:0.0:1789760774.367410:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.367432:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:0.0:1789760774.367434:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538289152:0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.367453:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538287744:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760774.367461:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.367494:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288384 00000100:00100000:0.0:1789760774.367498:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288384 00000100:00100000:0.0:1789760774.367525:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288128 00000100:00100000:0.0:1789760774.367528:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288128 00002000:00000024:3.0:1789760774.367552:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760774.367558:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760774.367567:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538288384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760774.367606:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538287744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 155us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.367614:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:0.0:1789760774.367624:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538288384:12345-0@lo:400:kworker.0 Request processed in 57us (2698us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.367633:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760774.367651:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1789760774.367658:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538288128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367686:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538288128:12345-0@lo:400:kworker.0 Request processed in 29us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.367726:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1789760774.367749:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288768 00000100:00100000:0.0:1789760774.367753:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288768 00000100:00100000:0.0:1789760774.367781:0:8086:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289280 00000100:00100000:0.0:1789760774.367784:0:8086:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289280 00000100:00100000:0.0:1789760774.367802:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:0.0:1789760774.367809:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538288768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367839:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538288768:12345-0@lo:400:kworker.0 Request processed in 30us (581us total) trans 0 rc 0/0 00000100:00080000:3.0:1789760774.367911:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760774.367917:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:0.0:1789760774.367933:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1789760774.367940:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538289280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.367969:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538289280:12345-0@lo:400:kworker.0 Request processed in 30us (670us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.367978:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1789760774.368003:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288256 00000100:00100000:0.0:1789760774.368007:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288256 00000100:00100000:0.0:1789760774.368037:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538288640 00000100:00100000:0.0:1789760774.368040:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538288640 00000100:00100000:0.0:1789760774.368062:0:14732:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289152 00000100:00100000:0.0:1789760774.368065:0:14732:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289152 00000100:00100000:0.0:1789760774.368087:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 955 00000100:00100000:0.0:1789760774.368100:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538288256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368131:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3464:x1876699538288256:12345-0@lo:400:kworker.0 Request processed in 38us (915us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.368137:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 955 00000100:00100000:0.0:1789760774.368150:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 956 00000100:00100000:0.0:1789760774.368156:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538288640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368182:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108016d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3464:x1876699538288640:12345-0@lo:400:kworker.0 Request processed in 26us (772us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.368188:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 956 00000100:00100000:0.0:1789760774.368201:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 957 00000100:00100000:0.0:1789760774.368209:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538289152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368236:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3464:x1876699538289152:12345-0@lo:400:kworker.0 Request processed in 29us (779us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.368243:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 957 00000100:00080000:0.0:1789760774.368273:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760774.368310:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538288384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368341:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538288128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368352:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3464:1876699538288640:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368361:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3464:1876699538289152:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368375:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234738e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538287744:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760774.368383:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3464:1876699538288256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368392:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538288768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760774.368400:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538289280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.368487:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329536 00000100:00100000:1.0:1789760774.368491:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329536 00000100:00100000:3.0:1789760774.368589:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755556597099 time 755604394162 state 0x0 ref 1 00000100:00100000:3.0:1789760774.368603:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755604654578 time 755604394162 state 0x1 ref 1 00000100:00100000:3.0:1789760774.368635:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755604654578 time 755683937526 state 0x0 ref 1 00000100:00100000:3.0:1789760774.368648:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507329536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760774.368690:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22216704, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:3.0:1789760774.368731:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.368746:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760774.368750:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.368757:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.368772:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.411615:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00002000:00010000:2.0:1789760774.411904:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22216704 -> 22220800 00010000:00010000:2.0:1789760774.411929:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.411942:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.411950:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.411958:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.411962:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.411972:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.411996:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.412004:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b908b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.412106:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507329536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43456us (50814us total) trans 4294972744 rc 0/0 00000100:00100000:2.0:1789760774.412115:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5454 00000100:00100000:2.0:1789760774.412125:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755604654578 time 755683937526 state 0x2 ref 0 00000100:00100000:3.0:1789760774.415846:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.415957:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329664 00000100:00100000:2.0:1789760774.415967:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329664 00000100:00100000:2.0:1789760774.416028:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755604654578 time 755683937526 state 0x0 ref 1 00000100:00100000:2.0:1789760774.416043:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755684197942 time 755683937526 state 0x1 ref 1 00000100:00100000:0.0:1789760774.417133:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755684197942 time 755732429216 state 0x0 ref 1 00000100:00100000:0.0:1789760774.417160:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760774.417218:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22220800, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:0.0:1789760774.417228:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.417245:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760774.417250:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.417257:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.417273:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.455277:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:2.0:1789760774.455401:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:2.0:1789760774.455416:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455429:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455439:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455446:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.455451:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455461:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455480:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.455489:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9092 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.455589:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38430us (39741us total) trans 4294972745 rc 0/0 00000100:00100000:2.0:1789760774.455599:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5455 00000100:00100000:2.0:1789760774.455608:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755684197942 time 755732429216 state 0x2 ref 0 00000100:00100000:1.0:1789760774.459469:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.459555:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329792 00000100:00100000:2.0:1789760774.459566:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329792 00000100:00100000:2.0:1789760774.459619:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755684197942 time 755732429216 state 0x0 ref 1 00000100:00100000:2.0:1789760774.459634:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755732689632 time 755732429216 state 0x1 ref 1 00000100:00100000:2.0:1789760774.459665:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755732689632 time 755774967930 state 0x0 ref 1 00000100:00100000:2.0:1789760774.459677:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507329792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.459766:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22224896, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.459776:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.459793:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760774.459798:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.459805:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.459819:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.492326:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00002000:00010000:2.0:1789760774.492374:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:2.0:1789760774.492389:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492402:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492411:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492417:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.492422:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492431:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492451:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.492460:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9099 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.492558:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507329792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32868us (33082us total) trans 4294972746 rc 0/0 00000100:00100000:2.0:1789760774.492567:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5456 00000100:00100000:2.0:1789760774.492576:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755732689632 time 755774967930 state 0x2 ref 0 00000100:00100000:0.0:1789760774.495967:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.496049:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507329920 00000100:00100000:2.0:1789760774.496059:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507329920 00000100:00100000:2.0:1789760774.496113:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755732689632 time 755774967930 state 0x0 ref 1 00000100:00100000:2.0:1789760774.496131:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755775228346 time 755774967930 state 0x1 ref 1 00000100:00100000:2.0:1789760774.496164:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755775228346 time 755811466734 state 0x0 ref 1 00000100:00100000:2.0:1789760774.496181:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.496231:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22228992, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.496240:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.496255:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760774.496260:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.496266:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.496286:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.544469:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:2.0:1789760774.544656:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:2.0:1789760774.544671:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544684:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544723:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544731:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.544735:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544744:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544764:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.544771:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.544869:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507329920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48689us (48905us total) trans 4294972747 rc 0/0 00000100:00100000:2.0:1789760774.544879:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5457 00000100:00100000:2.0:1789760774.544890:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755775228346 time 755811466734 state 0x2 ref 0 00000100:00100000:3.0:1789760774.550735:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.550761:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330048 00000100:00100000:2.0:1789760774.550770:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330048 00000100:00100000:2.0:1789760774.550837:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755775228346 time 755811466734 state 0x0 ref 1 00000100:00100000:2.0:1789760774.550852:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755811727150 time 755811466734 state 0x1 ref 1 00000100:00100000:2.0:1789760774.550885:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755811727150 time 755866187809 state 0x0 ref 1 00000100:00100000:2.0:1789760774.550898:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.550943:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22233088, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.550952:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.550967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760774.550972:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.550978:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.551005:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.604412:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:3.0:1789760774.604430:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604443:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604452:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604459:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.604464:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604472:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604492:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.604499:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b90a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.604605:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507330048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53705us (54027us total) trans 4294972748 rc 0/0 00000100:00100000:3.0:1789760774.604617:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5458 00000100:00100000:3.0:1789760774.604627:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755811727150 time 755866187809 state 0x2 ref 0 00000001:00080000:2.0:1789760774.604735:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00000100:00100000:1.0:1789760774.608906:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.609229:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330176 00000100:00100000:2.0:1789760774.609239:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330176 00000100:00100000:2.0:1789760774.609290:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755811727150 time 755866187809 state 0x0 ref 1 00000100:00100000:2.0:1789760774.609304:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755866448225 time 755866187809 state 0x1 ref 1 00000100:00100000:2.0:1789760774.609372:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755866448225 time 755924674324 state 0x0 ref 1 00000100:00100000:2.0:1789760774.609384:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.609432:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22237184, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.609440:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.609456:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760774.609461:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.609467:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.609481:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.673873:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00002000:00010000:3.0:1789760774.674059:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22237184 -> 22241280 00010000:00010000:3.0:1789760774.674078:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674092:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674100:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674107:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.674112:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674123:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674293:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.674303:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b90ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.674443:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65054us (65539us total) trans 4294972749 rc 0/0 00000100:00100000:1.0:1789760774.674455:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5459 00000100:00100000:1.0:1789760774.674464:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755866448225 time 755924674324 state 0x2 ref 0 00000100:00100000:1.0:1789760774.678150:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.678265:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330304 00000100:00100000:2.0:1789760774.678280:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330304 00000100:00100000:2.0:1789760774.678343:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755866448225 time 755924674324 state 0x0 ref 1 00000100:00100000:2.0:1789760774.678360:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755924934740 time 755924674324 state 0x1 ref 1 00000100:00100000:2.0:1789760774.678393:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755924934740 time 755993695697 state 0x0 ref 1 00000100:00100000:2.0:1789760774.678407:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507330304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.678468:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22241280, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.678478:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.678492:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760774.678497:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.678504:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.678517:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.707949:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00002000:00010000:3.0:1789760774.707990:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22241280 -> 22245376 00010000:00010000:3.0:1789760774.708004:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708014:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708020:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708024:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.708027:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708034:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708051:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.708059:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b90b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.708207:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507330304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29740us (30009us total) trans 4294972750 rc 0/0 00000100:00100000:3.0:1789760774.708222:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5460 00000100:00100000:3.0:1789760774.708231:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755924934740 time 755993695697 state 0x2 ref 0 00000100:00100000:3.0:1789760774.711964:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760774.712073:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330432 00000100:00100000:0.0:1789760774.712083:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330432 00000100:00100000:0.0:1789760774.712137:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755924934740 time 755993695697 state 0x0 ref 1 00000100:00100000:0.0:1789760774.712163:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755993956113 time 755993695697 state 0x1 ref 1 00000100:00100000:0.0:1789760774.712193:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755993956113 time 756027496494 state 0x0 ref 1 00000100:00100000:0.0:1789760774.712206:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760774.712252:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22245376, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:0.0:1789760774.712261:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.712276:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760774.712279:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.712284:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.712294:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.750913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00002000:00010000:3.0:1789760774.751111:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22245376 -> 22249472 00010000:00010000:3.0:1789760774.751127:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751140:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751150:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751157:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.751163:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751172:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751192:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.751199:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b90bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.751293:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39084us (39334us total) trans 4294972751 rc 0/0 00000100:00100000:3.0:1789760774.751303:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5461 00000100:00100000:3.0:1789760774.751312:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755993956113 time 756027496494 state 0x2 ref 0 00000100:00100000:1.0:1789760774.758597:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.758688:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330560 00000100:00100000:2.0:1789760774.758738:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330560 00000100:00100000:2.0:1789760774.758793:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 755993956113 time 756027496494 state 0x0 ref 1 00000100:00100000:2.0:1789760774.758810:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756027756910 time 756027496494 state 0x1 ref 1 00000100:00100000:2.0:1789760774.758839:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756027756910 time 756074142526 state 0x0 ref 1 00000100:00100000:2.0:1789760774.758851:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507330560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.758898:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22249472, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.758907:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.758923:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760774.758928:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.758934:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.758948:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.789604:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:3.0:1789760774.789624:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789638:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789648:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789656:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760774.789660:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789671:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789691:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.789744:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b90c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.789833:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507330560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30979us (31240us total) trans 4294972752 rc 0/0 00000100:00100000:3.0:1789760774.789843:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5462 00000100:00100000:3.0:1789760774.789853:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756027756910 time 756074142526 state 0x2 ref 0 00000001:00080000:0.0:1789760774.790387:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00000100:00100000:1.0:1789760774.794402:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.794508:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330688 00000100:00100000:2.0:1789760774.794519:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330688 00000100:00100000:2.0:1789760774.794587:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756027756910 time 756074142526 state 0x0 ref 1 00000100:00100000:2.0:1789760774.794602:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756074402942 time 756074142526 state 0x1 ref 1 00000100:00100000:2.0:1789760774.794634:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756074402942 time 756109937678 state 0x0 ref 1 00000100:00100000:2.0:1789760774.794649:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.794733:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22253568, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.794743:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.794760:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760774.794765:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.794772:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.794787:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.846229:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:2.0:1789760774.846376:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:2.0:1789760774.846391:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846403:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846412:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846419:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760774.846424:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846432:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846455:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.846462:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.846563:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51913us (52166us total) trans 4294972753 rc 0/0 00000100:00100000:2.0:1789760774.846573:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5463 00000100:00100000:2.0:1789760774.846582:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756074402942 time 756109937678 state 0x2 ref 0 00000100:00100000:3.0:1789760774.854918:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760774.855082:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330816 00000100:00100000:0.0:1789760774.855092:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330816 00000100:00100000:0.0:1789760774.855167:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756074402942 time 756109937678 state 0x0 ref 1 00000100:00100000:0.0:1789760774.855185:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756110198094 time 756109937678 state 0x1 ref 1 00000100:00100000:0.0:1789760774.855215:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756110198094 time 756170518259 state 0x0 ref 1 00000100:00100000:0.0:1789760774.855228:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507330816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760774.855274:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22257664, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:0.0:1789760774.855282:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.855298:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760774.855303:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.855309:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.855322:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.897356:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:0.0:1789760774.897376:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897390:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897400:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897413:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760774.897419:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897428:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897449:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897457:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.897552:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507330816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42310us (42627us total) trans 4294972754 rc 0/0 00000100:00100000:0.0:1789760774.897562:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5464 00000100:00100000:0.0:1789760774.897572:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756110198094 time 756170518259 state 0x2 ref 0 00000001:00080000:0.0:1789760774.898077:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00000100:00100000:1.0:1789760774.905858:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760774.906186:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507330944 00000100:00100000:3.0:1789760774.906196:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507330944 00000100:00100000:3.0:1789760774.906250:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756110198094 time 756170518259 state 0x0 ref 1 00000100:00100000:3.0:1789760774.906265:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756170778675 time 756170518259 state 0x1 ref 1 00000100:00100000:3.0:1789760774.906296:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756170778675 time 756221599440 state 0x0 ref 1 00000100:00100000:3.0:1789760774.906308:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760774.906358:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22261760, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:3.0:1789760774.906367:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.906384:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760774.906389:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.906396:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.906413:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.938453:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00002000:00010000:0.0:1789760774.938515:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:0.0:1789760774.938531:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938553:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938568:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938576:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760774.938580:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938590:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938609:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.938617:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b90d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.938759:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507330944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32447us (32908us total) trans 4294972755 rc 0/0 00000100:00100000:0.0:1789760774.938769:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5465 00000100:00100000:0.0:1789760774.938779:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756170778675 time 756221599440 state 0x2 ref 0 00000100:00100000:1.0:1789760774.941483:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760774.941597:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331072 00000100:00100000:2.0:1789760774.941607:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331072 00000100:00100000:2.0:1789760774.941659:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756170778675 time 756221599440 state 0x0 ref 1 00000100:00100000:2.0:1789760774.941674:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756221859856 time 756221599440 state 0x1 ref 1 00000100:00100000:2.0:1789760774.942082:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756221859856 time 756257383859 state 0x0 ref 1 00000100:00100000:2.0:1789760774.942099:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760774.942153:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22265856, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:2.0:1789760774.942162:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.942178:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760774.942183:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.942190:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.942203:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.989215:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00002000:00010000:0.0:1789760774.989301:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:0.0:1789760774.989318:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989336:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989346:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989352:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760774.989357:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989366:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989385:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989393:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b90df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.989841:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47741us (48362us total) trans 4294972756 rc 0/0 00000100:00100000:0.0:1789760774.989853:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5466 00000100:00100000:0.0:1789760774.989862:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756221859856 time 756257383859 state 0x2 ref 0 00000100:00100000:3.0:1789760774.997465:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760774.997570:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331200 00000100:00100000:0.0:1789760774.997580:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331200 00000100:00100000:0.0:1789760774.997637:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756221859856 time 756257383859 state 0x0 ref 1 00000100:00100000:0.0:1789760774.997654:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756257644275 time 756257383859 state 0x1 ref 1 00000100:00100000:0.0:1789760774.997684:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756257644275 time 756312987767 state 0x0 ref 1 00000100:00100000:0.0:1789760774.997844:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760774.998010:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22269952, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb007 00010000:00010000:0.0:1789760774.998021:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.998036:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760774.998041:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.998058:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.998073:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.045636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00002000:00010000:0.0:1789760775.046099:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22269952 -> 22274048 00002000:00010000:0.0:1789760775.046105:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:0.0:1789760775.046108:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:0.0:1789760775.046121:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046135:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046145:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046152:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760775.046157:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046166:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046187:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.046195:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.046323:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48481us (48864us total) trans 4294972757 rc 0/0 00000100:00100000:0.0:1789760775.046334:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5467 00000100:00100000:0.0:1789760775.046343:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756257644275 time 756312987767 state 0x2 ref 0 00000100:00100000:1.0:1789760775.052295:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760775.052359:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331328 00000100:00100000:0.0:1789760775.052375:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331328 00000100:00100000:0.0:1789760775.052437:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756257644275 time 756312987767 state 0x0 ref 1 00000100:00100000:0.0:1789760775.052450:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756313248183 time 756312987767 state 0x1 ref 1 00000100:00100000:0.0:1789760775.052549:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756313248183 time 756367840175 state 0x0 ref 1 00000100:00100000:0.0:1789760775.052563:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760775.052611:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22274048, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:0.0:1789760775.052620:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.052636:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760775.052732:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.052740:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.052755:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760775.052950:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760775.067845:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760775.094036:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00002000:00010000:0.0:1789760775.094135:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22274048 -> 22278144 00010000:00010000:0.0:1789760775.094151:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094163:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094173:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094180:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760775.094186:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094195:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094215:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.094222:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.094315:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41750us (42025us total) trans 4294972758 rc 0/0 00000100:00100000:0.0:1789760775.094326:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5468 00000100:00100000:0.0:1789760775.094335:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756313248183 time 756367840175 state 0x2 ref 0 00000100:00100000:1.0:1789760775.097983:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760775.098124:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331456 00000100:00100000:0.0:1789760775.098134:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331456 00000100:00100000:0.0:1789760775.098191:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756313248183 time 756367840175 state 0x0 ref 1 00000100:00100000:0.0:1789760775.098205:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756368100591 time 756367840175 state 0x1 ref 1 00000100:00100000:0.0:1789760775.098442:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756368100591 time 756413743967 state 0x0 ref 1 00000100:00100000:0.0:1789760775.098456:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760775.098515:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22278144, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:0.0:1789760775.098525:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.098549:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760775.098554:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.098560:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.098582:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.127104:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00002000:00010000:0.0:1789760775.127146:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22278144 -> 22282240 00010000:00010000:0.0:1789760775.127161:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127174:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127182:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127189:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760775.127195:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127204:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127230:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.127238:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b90f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.127328:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28870us (29350us total) trans 4294972759 rc 0/0 00000100:00100000:0.0:1789760775.127339:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5469 00000100:00100000:0.0:1789760775.127348:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756368100591 time 756413743967 state 0x2 ref 0 00000100:00100000:3.0:1789760775.130344:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760775.130454:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331584 00000100:00100000:0.0:1789760775.130464:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331584 00000100:00100000:0.0:1789760775.130518:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756368100591 time 756413743967 state 0x0 ref 1 00000100:00100000:0.0:1789760775.130534:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756414004383 time 756413743967 state 0x1 ref 1 00000100:00100000:0.0:1789760775.130575:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756414004383 time 756445878561 state 0x0 ref 1 00000100:00100000:0.0:1789760775.130587:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760775.130636:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22282240, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:0.0:1789760775.130645:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.130661:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760775.130666:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.130673:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.130687:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.181859:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:1.0:1789760775.181879:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181892:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181901:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181908:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.181913:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181922:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181939:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.181950:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b90fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.182384:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507331584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51792us (52042us total) trans 4294972760 rc 0/0 00000100:00100000:1.0:1789760775.182394:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5470 00000100:00100000:1.0:1789760775.182404:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756414004383 time 756445878561 state 0x2 ref 0 00000001:00080000:3.0:1789760775.183661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00000100:00100000:0.0:1789760775.187828:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.188052:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331712 00000100:00100000:3.0:1789760775.188063:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331712 00000100:00100000:3.0:1789760775.188287:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756414004383 time 756445878561 state 0x0 ref 1 00000100:00100000:3.0:1789760775.188302:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756446138977 time 756445878561 state 0x1 ref 1 00000100:00100000:3.0:1789760775.188373:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756446138977 time 756503674655 state 0x0 ref 1 00000100:00100000:3.0:1789760775.188387:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.188436:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22286336, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.188445:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.188470:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760775.188475:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.188482:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.188503:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.222217:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00002000:00010000:1.0:1789760775.222422:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:1.0:1789760775.222475:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222485:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222491:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222495:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.222500:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222509:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222640:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.222649:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9102 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.222779:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34390us (34956us total) trans 4294972761 rc 0/0 00000100:00100000:1.0:1789760775.222790:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5471 00000100:00100000:1.0:1789760775.222800:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756446138977 time 756503674655 state 0x2 ref 0 00000100:00100000:0.0:1789760775.230296:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.230412:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331840 00000100:00100000:2.0:1789760775.230421:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331840 00000100:00100000:2.0:1789760775.230469:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756446138977 time 756503674655 state 0x0 ref 1 00000100:00100000:2.0:1789760775.230480:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756503935071 time 756503674655 state 0x1 ref 1 00000100:00100000:2.0:1789760775.230503:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756503935071 time 756545807045 state 0x0 ref 1 00000100:00100000:2.0:1789760775.230515:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.230586:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22290432, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.230595:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.230610:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.230615:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.230622:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.230635:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.258207:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:2.0:1789760775.258315:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:2.0:1789760775.258332:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258346:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258355:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258363:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760775.258368:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258377:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258397:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258405:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9109 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.258558:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507331840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28039us (28267us total) trans 4294972762 rc 0/0 00000100:00100000:2.0:1789760775.258568:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5472 00000100:00100000:2.0:1789760775.258578:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756503935071 time 756545807045 state 0x2 ref 0 00000100:00100000:3.0:1789760775.265217:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.266072:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507331968 00000100:00100000:1.0:1789760775.266084:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507331968 00000100:00100000:1.0:1789760775.266217:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756503935071 time 756545807045 state 0x0 ref 1 00000100:00100000:1.0:1789760775.266231:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756546067461 time 756545807045 state 0x1 ref 1 00000100:00100000:1.0:1789760775.266262:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756546067461 time 756581565166 state 0x0 ref 1 00000100:00100000:1.0:1789760775.266274:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.266320:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22294528, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.266328:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.266351:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760775.266355:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.266362:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.266375:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.295851:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:2.0:1789760775.295891:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:2.0:1789760775.295906:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295920:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295933:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295941:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760775.295946:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295955:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295974:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.295993:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9110 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.296079:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507331968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29803us (30866us total) trans 4294972763 rc 0/0 00000100:00100000:2.0:1789760775.296089:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5473 00000100:00100000:2.0:1789760775.296099:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756546067461 time 756581565166 state 0x2 ref 0 00000100:00100000:0.0:1789760775.298529:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.298892:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332096 00000100:00100000:1.0:1789760775.298905:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332096 00000100:00100000:1.0:1789760775.298950:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756546067461 time 756581565166 state 0x0 ref 1 00000100:00100000:1.0:1789760775.298962:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756581825582 time 756581565166 state 0x1 ref 1 00000100:00100000:1.0:1789760775.298997:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756581825582 time 756614299720 state 0x0 ref 1 00000100:00100000:1.0:1789760775.299009:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.299056:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22298624, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.299095:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.299110:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760775.299114:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.299120:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.299138:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.333019:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00002000:00010000:3.0:1789760775.333357:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:3.0:1789760775.333372:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333385:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333394:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333400:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.333405:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333414:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333434:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.333446:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9117 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.333547:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34526us (35013us total) trans 4294972764 rc 0/0 00000100:00100000:3.0:1789760775.333557:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5474 00000100:00100000:3.0:1789760775.333568:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756581825582 time 756614299720 state 0x2 ref 0 00000100:00100000:0.0:1789760775.337154:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.337402:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332224 00000100:00100000:2.0:1789760775.337411:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332224 00000100:00100000:2.0:1789760775.337465:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756581825582 time 756614299720 state 0x0 ref 1 00000100:00100000:2.0:1789760775.337480:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756614560136 time 756614299720 state 0x1 ref 1 00000100:00100000:2.0:1789760775.337571:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756614560136 time 756652873304 state 0x0 ref 1 00000100:00100000:2.0:1789760775.337586:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.337632:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22302720, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.337641:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.337658:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.337662:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.337668:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.337682:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.372346:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00002000:00010000:2.0:1789760775.372475:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22302720 -> 22306816 00010000:00010000:2.0:1789760775.372490:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372507:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372516:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372543:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760775.372548:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372558:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372585:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.372593:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b911e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.372722:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35107us (35544us total) trans 4294972765 rc 0/0 00000100:00100000:2.0:1789760775.372732:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5475 00000100:00100000:2.0:1789760775.372742:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756614560136 time 756652873304 state 0x2 ref 0 00000100:00100000:0.0:1789760775.378749:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.378843:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332352 00000100:00100000:2.0:1789760775.378854:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332352 00000100:00100000:2.0:1789760775.378908:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756614560136 time 756652873304 state 0x0 ref 1 00000100:00100000:2.0:1789760775.378927:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756653133720 time 756652873304 state 0x1 ref 1 00000100:00100000:2.0:1789760775.378951:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756653133720 time 756694254881 state 0x0 ref 1 00000100:00100000:2.0:1789760775.378963:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.379008:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22306816, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.379017:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.379034:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.379039:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.379046:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.379060:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.387940:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760775.387963:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538289664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760775.388030:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760775.388144:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289664 00000100:00100000:3.0:1789760775.388151:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289664 00000100:00100000:3.0:1789760775.388205:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789760775.388215:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538289664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760775.388235:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3835691008 avail 3817865216 00000100:00100000:3.0:1789760775.388281:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538289664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (251us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760775.388289:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789760775.388333:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538289664:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760775.399488:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760775.399510:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538289792:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760775.399643:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760775.401274:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289792 00000100:00100000:0.0:1789760775.401288:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289792 00000100:00100000:0.0:1789760775.401956:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:0.0:1789760775.401969:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538289792:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760775.401995:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3835691008 avail 3817865216 00000100:00100000:0.0:1789760775.402071:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b45aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538289792:12345-0@lo:13:osp-pre-0-1.0 Request processed in 106us (2429us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760775.402079:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760775.402180:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538289792:0@lo:13:osp-pre-0-1.0 00002000:00010000:2.0:1789760775.419200:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22306816 -> 22310912 00010000:00010000:2.0:1789760775.419222:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419236:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419246:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419254:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760775.419259:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419268:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419288:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.419296:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9125 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.419391:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507332352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40424us (40754us total) trans 4294972766 rc 0/0 00000100:00100000:2.0:1789760775.419401:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5476 00000100:00100000:2.0:1789760775.419412:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756653133720 time 756694254881 state 0x2 ref 0 00000001:00080000:1.0:1789760775.420115:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00000100:00100000:1.0:1789760775.431050:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.431153:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332480 00000100:00100000:3.0:1789760775.431163:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332480 00000100:00100000:3.0:1789760775.431216:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756653133720 time 756694254881 state 0x0 ref 1 00000100:00100000:3.0:1789760775.431231:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756694515297 time 756694254881 state 0x1 ref 1 00000100:00100000:3.0:1789760775.431267:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756694515297 time 756746569635 state 0x0 ref 1 00000100:00100000:3.0:1789760775.431283:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.431331:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22310912, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.431340:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.431356:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760775.431361:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.431367:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.431380:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.465290:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00002000:00010000:3.0:1789760775.465423:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:3.0:1789760775.465438:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465450:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465459:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465465:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.465470:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465480:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465498:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.465507:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b912c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.465911:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34625us (34863us total) trans 4294972767 rc 0/0 00000100:00100000:3.0:1789760775.465921:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5477 00000100:00100000:3.0:1789760775.465930:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756694515297 time 756746569635 state 0x2 ref 0 00000100:00100000:1.0:1789760775.471377:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.471464:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332608 00000100:00100000:2.0:1789760775.471474:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332608 00000100:00100000:2.0:1789760775.471526:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756694515297 time 756746569635 state 0x0 ref 1 00000100:00100000:2.0:1789760775.471552:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756746830051 time 756746569635 state 0x1 ref 1 00000100:00100000:2.0:1789760775.471670:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756746830051 time 756786971895 state 0x0 ref 1 00000100:00100000:2.0:1789760775.471684:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.471827:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22315008, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.471836:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.471852:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.471856:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.471862:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.471876:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.501225:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22315008 -> 22319104 00000001:00080000:2.0:1789760775.501225:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00010000:00010000:0.0:1789760775.501241:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501254:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501264:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501272:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760775.501276:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501285:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501305:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.501313:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9133 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.501401:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29715us (30030us total) trans 4294972768 rc 0/0 00000100:00100000:0.0:1789760775.501411:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5478 00000100:00100000:0.0:1789760775.501420:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756746830051 time 756786971895 state 0x2 ref 0 00000100:00100000:3.0:1789760775.508005:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.508783:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332736 00000100:00100000:2.0:1789760775.508794:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332736 00000100:00100000:2.0:1789760775.509058:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756746830051 time 756786971895 state 0x0 ref 1 00000100:00100000:2.0:1789760775.509073:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756787232311 time 756786971895 state 0x1 ref 1 00000100:00100000:2.0:1789760775.509109:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756787232311 time 756824412198 state 0x0 ref 1 00000100:00100000:2.0:1789760775.509122:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.509174:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22319104, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.509183:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.509204:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.509209:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.509216:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.509229:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.546558:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00002000:00010000:3.0:1789760775.546741:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22319104 -> 22323200 00010000:00010000:3.0:1789760775.546758:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546772:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546781:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546795:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.546801:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546823:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546843:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.546850:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b913a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.546940:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507332736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37815us (38993us total) trans 4294972769 rc 0/0 00000100:00100000:3.0:1789760775.546950:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5479 00000100:00100000:3.0:1789760775.546960:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756787232311 time 756824412198 state 0x2 ref 0 00000100:00100000:1.0:1789760775.551463:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.551548:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332864 00000100:00100000:3.0:1789760775.551558:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332864 00000100:00100000:3.0:1789760775.551621:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756787232311 time 756824412198 state 0x0 ref 1 00000100:00100000:3.0:1789760775.551636:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756824672614 time 756824412198 state 0x1 ref 1 00000100:00100000:3.0:1789760775.551666:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756824672614 time 756866969144 state 0x0 ref 1 00000100:00100000:3.0:1789760775.551683:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.551769:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22323200, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.551779:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.551795:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760775.551800:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.551950:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.551968:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.597334:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00002000:00010000:1.0:1789760775.597447:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:1.0:1789760775.597462:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597476:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597485:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597492:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.597497:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597510:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597530:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.597547:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9141 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.597770:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46084us (46313us total) trans 4294972770 rc 0/0 00000100:00100000:1.0:1789760775.597781:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5480 00000100:00100000:1.0:1789760775.597791:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756824672614 time 756866969144 state 0x2 ref 0 00000100:00100000:1.0:1789760775.612078:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760775.612180:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507332992 00000100:00100000:0.0:1789760775.612190:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507332992 00000100:00100000:0.0:1789760775.612244:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756824672614 time 756866969144 state 0x0 ref 1 00000100:00100000:0.0:1789760775.612259:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756867229560 time 756866969144 state 0x1 ref 1 00000100:00100000:0.0:1789760775.612289:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756867229560 time 756927593063 state 0x0 ref 1 00000100:00100000:0.0:1789760775.612301:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760775.612348:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22327296, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:0.0:1789760775.612358:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.612374:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760775.612378:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.612384:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.612397:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.645977:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:2.0:1789760775.646082:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22327296 -> 22331392 00010000:00010000:2.0:1789760775.646098:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646111:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646120:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646128:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760775.646134:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646149:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646169:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.646177:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9148 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.646310:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507332992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34005us (34236us total) trans 4294972771 rc 0/0 00000100:00100000:2.0:1789760775.646322:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5481 00000100:00100000:2.0:1789760775.646333:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756867229560 time 756927593063 state 0x2 ref 0 00000100:00100000:2.0:1789760775.650406:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.650499:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333120 00000100:00100000:3.0:1789760775.650509:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333120 00000100:00100000:3.0:1789760775.650681:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756867229560 time 756927593063 state 0x0 ref 1 00000100:00100000:3.0:1789760775.651970:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756927853479 time 756927593063 state 0x1 ref 1 00000100:00100000:2.0:1789760775.652097:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756927853479 time 756967397655 state 0x0 ref 1 00000100:00100000:2.0:1789760775.652120:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.652173:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22331392, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.652183:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.652199:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760775.652204:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.652210:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.652223:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.683398:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:1.0:1789760775.683420:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683433:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683442:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683454:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.683460:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683470:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683491:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683500:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b914f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.683748:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507333120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31633us (33347us total) trans 4294972772 rc 0/0 00000100:00100000:1.0:1789760775.683759:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5482 00000100:00100000:1.0:1789760775.683769:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756927853479 time 756967397655 state 0x2 ref 0 00000001:00080000:1.0:1789760775.683810:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00000100:00100000:1.0:1789760775.688933:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.689126:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333248 00000100:00100000:3.0:1789760775.689137:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333248 00000100:00100000:3.0:1789760775.689190:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756927853479 time 756967397655 state 0x0 ref 1 00000100:00100000:3.0:1789760775.689206:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756967658071 time 756967397655 state 0x1 ref 1 00000100:00100000:3.0:1789760775.689242:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756967658071 time 757004545756 state 0x0 ref 1 00000100:00100000:3.0:1789760775.689258:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.689305:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22335488, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.689313:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689329:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760775.689333:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.689338:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689352:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.710835:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00002000:00010000:1.0:1789760775.711043:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:1.0:1789760775.711060:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711074:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711086:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711094:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.711099:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711109:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711131:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.711139:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9156 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.711235:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21974us (22305us total) trans 4294972773 rc 0/0 00000100:00100000:1.0:1789760775.711246:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5483 00000100:00100000:1.0:1789760775.711256:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756967658071 time 757004545756 state 0x2 ref 0 00000100:00100000:0.0:1789760775.715074:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760775.715601:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333376 00000100:00100000:2.0:1789760775.715611:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333376 00000100:00100000:2.0:1789760775.715666:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 756967658071 time 757004545756 state 0x0 ref 1 00000100:00100000:2.0:1789760775.715683:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757004806172 time 757004545756 state 0x1 ref 1 00000100:00100000:2.0:1789760775.715751:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757004806172 time 757031053997 state 0x0 ref 1 00000100:00100000:2.0:1789760775.715766:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760775.715816:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22339584, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:2.0:1789760775.715824:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.715840:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760775.715845:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.715851:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.715865:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.743056:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00002000:00010000:1.0:1789760775.743222:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:1.0:1789760775.743238:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743268:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743279:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743287:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.743292:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743302:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743321:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.743396:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b915d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.743491:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27723us (28421us total) trans 4294972774 rc 0/0 00000100:00100000:1.0:1789760775.743501:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5484 00000100:00100000:1.0:1789760775.743511:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757004806172 time 757031053997 state 0x2 ref 0 00000100:00100000:2.0:1789760775.747055:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.747328:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333504 00000100:00100000:1.0:1789760775.747339:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333504 00000100:00100000:1.0:1789760775.747395:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757004806172 time 757031053997 state 0x0 ref 1 00000100:00100000:1.0:1789760775.747410:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757031314413 time 757031053997 state 0x1 ref 1 00000100:00100000:1.0:1789760775.747439:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757031314413 time 757062742237 state 0x0 ref 1 00000100:00100000:1.0:1789760775.747452:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.747618:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22343680, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.747629:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.747645:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.747650:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.747655:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.747670:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.773888:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00002000:00010000:1.0:1789760775.773981:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:1.0:1789760775.773999:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774012:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774027:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774034:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.774039:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774048:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774068:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.774076:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9164 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.774165:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26710us (27116us total) trans 4294972775 rc 0/0 00000100:00100000:1.0:1789760775.774177:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5485 00000100:00100000:1.0:1789760775.774187:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757031314413 time 757062742237 state 0x2 ref 0 00000100:00100000:2.0:1789760775.779779:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.780556:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333632 00000100:00100000:1.0:1789760775.780567:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333632 00000100:00100000:1.0:1789760775.780616:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757031314413 time 757062742237 state 0x0 ref 1 00000100:00100000:1.0:1789760775.780631:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757063002653 time 757062742237 state 0x1 ref 1 00000100:00100000:1.0:1789760775.780660:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757063002653 time 757095963116 state 0x0 ref 1 00000100:00100000:1.0:1789760775.780671:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.780736:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22347776, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.780745:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.780761:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.780765:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.780771:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.780786:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.807465:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00002000:00010000:1.0:1789760775.807640:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:1.0:1789760775.807656:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807668:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807677:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807685:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.807691:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807724:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807744:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.807752:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b916b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.807842:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27167us (28154us total) trans 4294972776 rc 0/0 00000100:00100000:1.0:1789760775.807853:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5486 00000100:00100000:1.0:1789760775.807864:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757063002653 time 757095963116 state 0x2 ref 0 00000100:00100000:2.0:1789760775.810438:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.810499:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333760 00000100:00100000:1.0:1789760775.810509:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333760 00000100:00100000:1.0:1789760775.810578:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757063002653 time 757095963116 state 0x0 ref 1 00000100:00100000:1.0:1789760775.810592:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757096223532 time 757095963116 state 0x1 ref 1 00000100:00100000:1.0:1789760775.810620:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757096223532 time 757125923626 state 0x0 ref 1 00000100:00100000:1.0:1789760775.810632:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.810677:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22351872, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.810687:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.810738:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.810742:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.810749:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.810763:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.838181:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00002000:00010000:1.0:1789760775.838305:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:1.0:1789760775.838318:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838328:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838334:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838339:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.838344:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838354:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838379:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.838387:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9172 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.838477:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27841us (28045us total) trans 4294972777 rc 0/0 00000100:00100000:1.0:1789760775.838485:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5487 00000100:00100000:1.0:1789760775.838495:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757096223532 time 757125923626 state 0x2 ref 0 00000100:00100000:2.0:1789760775.843855:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.843901:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507333888 00000100:00100000:1.0:1789760775.843911:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507333888 00000100:00100000:1.0:1789760775.843965:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757096223532 time 757125923626 state 0x0 ref 1 00000100:00100000:1.0:1789760775.843979:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757126184042 time 757125923626 state 0x1 ref 1 00000100:00100000:1.0:1789760775.844010:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757126184042 time 757159313417 state 0x0 ref 1 00000100:00100000:1.0:1789760775.844022:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507333888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.844067:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22355968, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.844076:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.844092:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.844096:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.844102:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.844116:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.864471:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:3.0:1789760775.864491:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864504:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864514:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864521:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.864526:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864547:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864578:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.864587:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9179 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.864678:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507333888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 20651us (20824us total) trans 4294972778 rc 0/0 00000100:00100000:3.0:1789760775.864688:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5488 00000100:00100000:3.0:1789760775.864742:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757126184042 time 757159313417 state 0x2 ref 0 00000001:00080000:0.0:1789760775.865215:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00000100:00100000:2.0:1789760775.870810:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.871007:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334016 00000100:00100000:1.0:1789760775.871016:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334016 00000100:00100000:1.0:1789760775.871071:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757126184042 time 757159313417 state 0x0 ref 1 00000100:00100000:1.0:1789760775.871087:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757159573833 time 757159313417 state 0x1 ref 1 00000100:00100000:1.0:1789760775.871157:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757159573833 time 757186459526 state 0x0 ref 1 00000100:00100000:1.0:1789760775.871170:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.871217:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22360064, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.871238:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.871356:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.871363:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.871371:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.871388:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.901244:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1083ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538289920:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760775.901311:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760775.901445:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538289920 00000100:00100000:3.0:1789760775.901453:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538289920 00000100:00100000:3.0:1789760775.901624:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1789760775.901634:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538289920:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760775.901669:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760775.901819:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538289920:12345-0@lo:41:osp-pre-1-0.0 Request processed in 185us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760775.901826:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1789760775.901839:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1083ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538289920:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760775.908316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00002000:00010000:3.0:1789760775.908561:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22360064 -> 22364160 00010000:00010000:3.0:1789760775.908578:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908591:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908602:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908610:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.908616:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908625:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908654:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.908663:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9180 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.908772:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37600us (37966us total) trans 4294972779 rc 0/0 00000100:00100000:3.0:1789760775.908783:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5489 00000100:00100000:3.0:1789760775.908793:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757159573833 time 757186459526 state 0x2 ref 0 00000100:00100000:2.0:1789760775.914416:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760775.914850:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334144 00000100:00100000:1.0:1789760775.914860:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334144 00000100:00100000:1.0:1789760775.914913:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757159573833 time 757186459526 state 0x0 ref 1 00000100:00100000:1.0:1789760775.914928:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757186719942 time 757186459526 state 0x1 ref 1 00000100:00100000:1.0:1789760775.914960:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757186719942 time 757230263167 state 0x0 ref 1 00000100:00100000:1.0:1789760775.914972:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760775.915018:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22364160, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:1.0:1789760775.915026:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.915042:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760775.915047:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.915053:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.915068:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.950507:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00002000:00010000:1.0:1789760775.950566:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22364160 -> 22368256 00010000:00010000:1.0:1789760775.950580:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950600:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950609:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950617:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760775.950621:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950631:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950652:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.950661:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9187 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.950792:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10819ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35817us (36378us total) trans 4294972780 rc 0/0 00000100:00100000:1.0:1789760775.950812:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5490 00000100:00100000:1.0:1789760775.950822:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757186719942 time 757230263167 state 0x2 ref 0 00000100:00100000:2.0:1789760775.957160:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.957234:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334272 00000100:00100000:3.0:1789760775.957245:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334272 00000100:00100000:3.0:1789760775.957300:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757186719942 time 757230263167 state 0x0 ref 1 00000100:00100000:3.0:1789760775.957315:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757230523583 time 757230263167 state 0x1 ref 1 00000100:00100000:3.0:1789760775.957340:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757230523583 time 757272643403 state 0x0 ref 1 00000100:00100000:3.0:1789760775.957352:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.957397:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22368256, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.957404:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.957417:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760775.957421:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.957425:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.957436:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.984336:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00002000:00010000:3.0:1789760775.984471:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:3.0:1789760775.984486:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984504:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984513:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984520:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760775.984524:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984545:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984564:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.984634:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b918e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.984811:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27453us (27654us total) trans 4294972781 rc 0/0 00000100:00100000:3.0:1789760775.984824:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5491 00000100:00100000:3.0:1789760775.984835:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757230523583 time 757272643403 state 0x2 ref 0 00000100:00100000:2.0:1789760775.988121:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760775.989052:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334400 00000100:00100000:3.0:1789760775.989065:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334400 00000100:00100000:3.0:1789760775.989120:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757230523583 time 757272643403 state 0x0 ref 1 00000100:00100000:3.0:1789760775.989134:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757272903819 time 757272643403 state 0x1 ref 1 00000100:00100000:3.0:1789760775.989207:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757272903819 time 757304509245 state 0x0 ref 1 00000100:00100000:3.0:1789760775.989219:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507334400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760775.989272:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22372352, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb007 00010000:00010000:3.0:1789760775.989282:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.989297:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760775.989302:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.989315:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.989331:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.010158:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00002000:00010000:1.0:1789760776.010223:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22372352 -> 22376448 00002000:00010000:1.0:1789760776.010228:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:1.0:1789760776.010231:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:1.0:1789760776.010244:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010256:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010265:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010271:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.010277:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010286:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010305:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.010313:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9195 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.010437:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507334400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21216us (22321us total) trans 4294972782 rc 0/0 00000100:00100000:1.0:1789760776.010450:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5492 00000100:00100000:1.0:1789760776.010460:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757272903819 time 757304509245 state 0x2 ref 0 00000100:00100000:2.0:1789760776.016051:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.016102:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334528 00000100:00100000:1.0:1789760776.016113:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334528 00000100:00100000:1.0:1789760776.016177:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757272903819 time 757304509245 state 0x0 ref 1 00000100:00100000:1.0:1789760776.016193:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757304769661 time 757304509245 state 0x1 ref 1 00000100:00100000:1.0:1789760776.016220:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757304769661 time 757331523332 state 0x0 ref 1 00000100:00100000:1.0:1789760776.016231:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507334528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.016281:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22376448, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.016289:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.016304:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.016309:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.016316:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.016330:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.049090:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:2.0:1789760776.049110:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049123:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049133:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049140:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.049145:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049153:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049174:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.049183:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b919c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.049283:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507334528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33049us (33234us total) trans 4294972783 rc 0/0 00000100:00100000:2.0:1789760776.049294:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5493 00000100:00100000:2.0:1789760776.049303:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757304769661 time 757331523332 state 0x2 ref 0 00000001:00080000:0.0:1789760776.049460:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00000100:00100000:2.0:1789760776.052954:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760776.053107:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334656 00000100:00100000:3.0:1789760776.053117:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334656 00000100:00100000:3.0:1789760776.053182:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757304769661 time 757331523332 state 0x0 ref 1 00000100:00100000:3.0:1789760776.053198:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757331783748 time 757331523332 state 0x1 ref 1 00000100:00100000:3.0:1789760776.053225:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757331783748 time 757368528914 state 0x0 ref 1 00000100:00100000:3.0:1789760776.053242:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507334656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760776.053289:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22380544, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:3.0:1789760776.053298:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.053313:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760776.053317:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.053322:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.053336:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.083279:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:1.0:1789760776.083297:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083311:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083320:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083327:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.083331:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083339:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083357:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083364:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b91a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.083453:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507334656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30209us (30503us total) trans 4294972784 rc 0/0 00000100:00100000:1.0:1789760776.083463:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5494 00000100:00100000:1.0:1789760776.083472:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757331783748 time 757368528914 state 0x2 ref 0 00000001:00080000:1.0:1789760776.083813:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00010000:00010000:3.0:1789760776.091847:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760776.094501:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760776.094787:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334784 00000100:00100000:3.0:1789760776.094796:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334784 00000100:00100000:3.0:1789760776.094852:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757331783748 time 757368528914 state 0x0 ref 1 00000100:00100000:3.0:1789760776.094871:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757368789330 time 757368528914 state 0x1 ref 1 00000100:00100000:3.0:1789760776.094909:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757368789330 time 757410212425 state 0x0 ref 1 00000100:00100000:3.0:1789760776.094923:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507334784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760776.094971:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22384640, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:3.0:1789760776.094988:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.095010:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760776.095015:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.095021:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.095037:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.144301:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:3.0:1789760776.144321:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144334:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144342:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144350:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760776.144355:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144365:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144384:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.144393:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.144487:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507334784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49562us (49990us total) trans 4294972785 rc 0/0 00000100:00100000:3.0:1789760776.144498:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5495 00000100:00100000:3.0:1789760776.144508:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757368789330 time 757410212425 state 0x2 ref 0 00000001:00080000:0.0:1789760776.144590:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00000100:00100000:2.0:1789760776.150585:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.150690:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507334912 00000100:00100000:1.0:1789760776.150734:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507334912 00000100:00100000:1.0:1789760776.150789:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757368789330 time 757410212425 state 0x0 ref 1 00000100:00100000:1.0:1789760776.150816:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757410472841 time 757410212425 state 0x1 ref 1 00000100:00100000:1.0:1789760776.150845:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757410472841 time 757466148093 state 0x0 ref 1 00000100:00100000:1.0:1789760776.150857:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507334912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.150907:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22388736, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.150916:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.150931:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.150936:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.150942:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.150957:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.186494:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00002000:00010000:1.0:1789760776.186612:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:1.0:1789760776.186628:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186642:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186652:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186659:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.186664:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186673:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186716:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.186724:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.186804:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507334912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35944us (36223us total) trans 4294972786 rc 0/0 00000100:00100000:1.0:1789760776.186814:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5496 00000100:00100000:1.0:1789760776.186824:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757410472841 time 757466148093 state 0x2 ref 0 00000100:00100000:2.0:1789760776.191242:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.191317:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335040 00000100:00100000:1.0:1789760776.191328:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335040 00000100:00100000:1.0:1789760776.191380:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757410472841 time 757466148093 state 0x0 ref 1 00000100:00100000:1.0:1789760776.191394:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757466408509 time 757466148093 state 0x1 ref 1 00000100:00100000:1.0:1789760776.191425:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757466408509 time 757506727869 state 0x0 ref 1 00000100:00100000:1.0:1789760776.191438:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.191494:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22392832, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.191503:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.191519:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.191524:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.191542:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.191558:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.252862:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00002000:00010000:0.0:1789760776.253029:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:0.0:1789760776.253049:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253066:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253075:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253084:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760776.253090:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253101:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253126:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.253134:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b91b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.253284:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61842us (62044us total) trans 4294972787 rc 0/0 00000100:00100000:0.0:1789760776.253296:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5497 00000100:00100000:0.0:1789760776.253308:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757466408509 time 757506727869 state 0x2 ref 0 00000100:00100000:2.0:1789760776.256817:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760776.260608:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335168 00000100:00100000:3.0:1789760776.260623:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335168 00000100:00100000:3.0:1789760776.260684:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757466408509 time 757506727869 state 0x0 ref 1 00000100:00100000:3.0:1789760776.260715:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757506988285 time 757506727869 state 0x1 ref 1 00000100:00100000:3.0:1789760776.260746:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757506988285 time 757576049418 state 0x0 ref 1 00000100:00100000:3.0:1789760776.260758:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760776.260806:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22396928, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:3.0:1789760776.260815:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.260832:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760776.260836:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.260842:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.260860:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.286871:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00002000:00010000:0.0:1789760776.286917:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:0.0:1789760776.286933:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286946:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286955:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286967:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760776.286971:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286980:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286997:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.287004:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b91bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.287089:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26329us (30281us total) trans 4294972788 rc 0/0 00000100:00100000:0.0:1789760776.287099:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5498 00000100:00100000:0.0:1789760776.287109:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757506988285 time 757576049418 state 0x2 ref 0 00000100:00100000:2.0:1789760776.293029:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.293195:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335296 00000100:00100000:1.0:1789760776.293205:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335296 00000100:00100000:1.0:1789760776.293256:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757506988285 time 757576049418 state 0x0 ref 1 00000100:00100000:1.0:1789760776.293270:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757576309834 time 757576049418 state 0x1 ref 1 00000100:00100000:1.0:1789760776.293300:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757576309834 time 757608603714 state 0x0 ref 1 00000100:00100000:1.0:1789760776.293311:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.293357:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22401024, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.293365:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.293380:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.293384:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.293391:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.293405:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.331993:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:1.0:1789760776.332214:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:1.0:1789760776.332230:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332242:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332250:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332258:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.332262:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332271:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332290:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.332298:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.332383:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39068us (39359us total) trans 4294972789 rc 0/0 00000100:00100000:1.0:1789760776.332393:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5499 00000100:00100000:1.0:1789760776.332403:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757576309834 time 757608603714 state 0x2 ref 0 00000100:00100000:3.0:1789760776.335992:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.336232:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335424 00000100:00100000:0.0:1789760776.336243:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335424 00000100:00100000:0.0:1789760776.336309:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757576309834 time 757608603714 state 0x0 ref 1 00000100:00100000:0.0:1789760776.336324:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757608864130 time 757608603714 state 0x1 ref 1 00000100:00100000:0.0:1789760776.336405:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757608864130 time 757651708015 state 0x0 ref 1 00000100:00100000:0.0:1789760776.336418:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.336461:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22405120, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.336470:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.336485:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760776.336489:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.336495:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.336510:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.401131:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00002000:00010000:0.0:1789760776.405449:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:0.0:1789760776.405470:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405484:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405494:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405502:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760776.405508:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405517:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405545:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.405553:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.405665:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69243us (69676us total) trans 4294972790 rc 0/0 00000100:00100000:0.0:1789760776.405675:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5500 00000100:00100000:0.0:1789760776.405684:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757608864130 time 757651708015 state 0x2 ref 0 00000100:00100000:0.0:1789760776.411935:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1083ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538290048:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760776.412008:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760776.412062:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290048 00000100:00100000:1.0:1789760776.412071:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290048 00000100:00100000:1.0:1789760776.412132:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:1.0:1789760776.412142:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538290048:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760776.412178:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760776.412231:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3462:x1876699538290048:12345-0@lo:41:osp-pre-0-1.0 Request processed in 90us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760776.412238:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:2.0:1789760776.412402:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1083ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538290048:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760776.413137:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.413272:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335552 00000100:00100000:0.0:1789760776.413279:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335552 00000100:00100000:0.0:1789760776.413334:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757608864130 time 757651708015 state 0x0 ref 1 00000100:00100000:0.0:1789760776.413348:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757651968431 time 757651708015 state 0x1 ref 1 00000100:00100000:0.0:1789760776.413376:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757651968431 time 757728679062 state 0x0 ref 1 00000100:00100000:0.0:1789760776.413387:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.413433:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22409216, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.413443:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413458:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760776.413463:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.413469:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413483:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.451029:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00002000:00010000:2.0:1789760776.451130:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:2.0:1789760776.451146:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451159:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451169:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451176:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.451181:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451191:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451210:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.451219:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b91d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.451346:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507335552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37955us (38207us total) trans 4294972791 rc 0/0 00000100:00100000:2.0:1789760776.451356:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5501 00000100:00100000:2.0:1789760776.451366:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757651968431 time 757728679062 state 0x2 ref 0 00000100:00100000:2.0:1789760776.461665:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.462174:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335680 00000100:00100000:1.0:1789760776.462184:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335680 00000100:00100000:1.0:1789760776.462250:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757651968431 time 757728679062 state 0x0 ref 1 00000100:00100000:1.0:1789760776.462265:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757728939478 time 757728679062 state 0x1 ref 1 00000100:00100000:1.0:1789760776.462295:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757728939478 time 757777598092 state 0x0 ref 1 00000100:00100000:1.0:1789760776.462318:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507335680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.462367:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22413312, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.462377:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.462393:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.462397:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.462404:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.462417:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.500438:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:2.0:1789760776.500737:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:2.0:1789760776.500754:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500772:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500780:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500786:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.500791:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500801:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500821:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.500829:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b91db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.500920:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507335680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38608us (39258us total) trans 4294972792 rc 0/0 00000100:00100000:2.0:1789760776.500930:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5502 00000100:00100000:2.0:1789760776.500940:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757728939478 time 757777598092 state 0x2 ref 0 00000100:00100000:3.0:1789760776.505589:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.505741:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335808 00000100:00100000:0.0:1789760776.505751:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335808 00000100:00100000:0.0:1789760776.505809:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757728939478 time 757777598092 state 0x0 ref 1 00000100:00100000:0.0:1789760776.505825:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757777858508 time 757777598092 state 0x1 ref 1 00000100:00100000:0.0:1789760776.505852:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757777858508 time 757821155814 state 0x0 ref 1 00000100:00100000:0.0:1789760776.505868:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507335808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.505918:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22417408, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.505927:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.505943:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760776.505948:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.505954:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.505968:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.544560:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00002000:00010000:1.0:1789760776.544715:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22417408 -> 22421504 00010000:00010000:1.0:1789760776.544730:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544743:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544752:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544759:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.544764:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544774:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544794:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.544801:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b91e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.544890:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507335808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39019us (39305us total) trans 4294972793 rc 0/0 00000100:00100000:1.0:1789760776.544900:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5503 00000100:00100000:1.0:1789760776.544911:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757777858508 time 757821155814 state 0x2 ref 0 00000100:00100000:1.0:1789760776.548659:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.548825:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507335936 00000100:00100000:0.0:1789760776.548836:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507335936 00000100:00100000:0.0:1789760776.548894:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757777858508 time 757821155814 state 0x0 ref 1 00000100:00100000:0.0:1789760776.548909:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757821416230 time 757821155814 state 0x1 ref 1 00000100:00100000:0.0:1789760776.548942:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757821416230 time 757864245001 state 0x0 ref 1 00000100:00100000:0.0:1789760776.548954:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507335936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.549013:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22421504, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.549022:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.549041:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760776.549045:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.549051:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.549064:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.580513:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:0.0:1789760776.580648:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22421504 -> 22425600 00010000:00010000:0.0:1789760776.580663:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580673:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580681:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580688:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760776.580715:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580725:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580750:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580757:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b91e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.580838:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507335936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31882us (32185us total) trans 4294972794 rc 0/0 00000100:00100000:0.0:1789760776.580847:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5504 00000100:00100000:0.0:1789760776.580856:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757821416230 time 757864245001 state 0x2 ref 0 00000100:00100000:2.0:1789760776.587497:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.587608:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336064 00000100:00100000:1.0:1789760776.587619:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336064 00000100:00100000:1.0:1789760776.587676:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757821416230 time 757864245001 state 0x0 ref 1 00000100:00100000:1.0:1789760776.587707:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757864505417 time 757864245001 state 0x1 ref 1 00000100:00100000:1.0:1789760776.587772:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757864505417 time 757903074814 state 0x0 ref 1 00000100:00100000:1.0:1789760776.587785:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.587834:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22425600, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.587845:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.587862:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.587866:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.587872:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.587886:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.616093:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00002000:00010000:2.0:1789760776.616317:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:2.0:1789760776.616331:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616344:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616354:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616361:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.616365:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616374:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616394:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.616402:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b91f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.616487:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28699us (28993us total) trans 4294972795 rc 0/0 00000100:00100000:2.0:1789760776.616499:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5505 00000100:00100000:2.0:1789760776.616509:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757864505417 time 757903074814 state 0x2 ref 0 00000100:00100000:0.0:1789760776.621296:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.621456:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336192 00000100:00100000:1.0:1789760776.621466:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336192 00000100:00100000:1.0:1789760776.621518:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757864505417 time 757903074814 state 0x0 ref 1 00000100:00100000:1.0:1789760776.621544:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757903335230 time 757903074814 state 0x1 ref 1 00000100:00100000:1.0:1789760776.621611:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757903335230 time 757936913916 state 0x0 ref 1 00000100:00100000:1.0:1789760776.621625:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.621674:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22429696, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.621683:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.621873:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760776.621878:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.621884:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.621900:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.657344:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00002000:00010000:1.0:1789760776.657425:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22429696 -> 22433792 00010000:00010000:1.0:1789760776.657440:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657453:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657463:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657470:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.657475:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657483:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657502:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.657510:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b91f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.658194:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36563us (36899us total) trans 4294972796 rc 0/0 00000100:00100000:1.0:1789760776.658206:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5506 00000100:00100000:1.0:1789760776.658217:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757903335230 time 757936913916 state 0x2 ref 0 00000100:00100000:1.0:1789760776.661632:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.662006:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336320 00000100:00100000:0.0:1789760776.662016:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336320 00000100:00100000:0.0:1789760776.662071:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757903335230 time 757936913916 state 0x0 ref 1 00000100:00100000:0.0:1789760776.662086:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757937174332 time 757936913916 state 0x1 ref 1 00000100:00100000:0.0:1789760776.662118:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757937174332 time 757977420675 state 0x0 ref 1 00000100:00100000:0.0:1789760776.662209:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.662258:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22433792, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.662268:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.662290:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760776.662295:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.662301:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.662316:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.701657:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:1.0:1789760776.701809:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:1.0:1789760776.701825:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701838:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701847:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701855:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.701860:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701870:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701892:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.701905:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b91fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.702035:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39826us (40407us total) trans 4294972797 rc 0/0 00000100:00100000:1.0:1789760776.702047:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5507 00000100:00100000:1.0:1789760776.702058:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757937174332 time 757977420675 state 0x2 ref 0 00000100:00100000:0.0:1789760776.705485:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760776.705926:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336448 00000100:00100000:2.0:1789760776.705938:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336448 00000100:00100000:2.0:1789760776.705997:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757937174332 time 757977420675 state 0x0 ref 1 00000100:00100000:2.0:1789760776.706075:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757977681091 time 757977420675 state 0x1 ref 1 00000100:00100000:2.0:1789760776.706150:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757977681091 time 758021452339 state 0x0 ref 1 00000100:00100000:2.0:1789760776.706164:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760776.706214:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22437888, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:2.0:1789760776.706223:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.706238:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760776.706243:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.706249:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.706264:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.749962:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00002000:00010000:2.0:1789760776.750680:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22437888 -> 22441984 00010000:00010000:2.0:1789760776.750729:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750749:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750759:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750766:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.750771:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750780:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750801:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.750810:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9205 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.750914:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44746us (45433us total) trans 4294972798 rc 0/0 00000100:00100000:2.0:1789760776.750925:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5508 00000100:00100000:2.0:1789760776.750933:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757977681091 time 758021452339 state 0x2 ref 0 00000100:00100000:0.0:1789760776.756040:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.756333:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336576 00000100:00100000:1.0:1789760776.756343:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336576 00000100:00100000:1.0:1789760776.756397:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 757977681091 time 758021452339 state 0x0 ref 1 00000100:00100000:1.0:1789760776.756415:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758021712755 time 758021452339 state 0x1 ref 1 00000100:00100000:1.0:1789760776.756544:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758021712755 time 758071845700 state 0x0 ref 1 00000100:00100000:1.0:1789760776.756559:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.756609:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22441984, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.756619:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.756634:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760776.756638:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.756645:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.756659:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.808727:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22441984 -> 22446080 00010000:00010000:1.0:1789760776.808746:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808759:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808769:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808776:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.808781:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808791:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808811:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.808824:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b920c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.808852:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00000100:00100000:1.0:1789760776.808923:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52363us (52889us total) trans 4294972799 rc 0/0 00000100:00100000:1.0:1789760776.808934:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5509 00000100:00100000:1.0:1789760776.808943:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758021712755 time 758071845700 state 0x2 ref 0 00000100:00100000:1.0:1789760776.813789:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760776.813829:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336704 00000100:00100000:0.0:1789760776.813840:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336704 00000100:00100000:0.0:1789760776.813895:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758021712755 time 758071845700 state 0x0 ref 1 00000100:00100000:0.0:1789760776.813910:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758072106116 time 758071845700 state 0x1 ref 1 00000100:00100000:0.0:1789760776.813943:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758072106116 time 758129245624 state 0x0 ref 1 00000100:00100000:0.0:1789760776.813956:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760776.814003:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22446080, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:0.0:1789760776.814013:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.814029:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760776.814034:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.814040:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.814053:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.874154:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00002000:00010000:2.0:1789760776.874493:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:2.0:1789760776.874508:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874520:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874550:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874672:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.874677:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874687:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874727:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.874735:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9213 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.874863:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60905us (61082us total) trans 4294972800 rc 0/0 00000100:00100000:2.0:1789760776.874874:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5510 00000100:00100000:2.0:1789760776.874884:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758072106116 time 758129245624 state 0x2 ref 0 00000100:00100000:0.0:1789760776.879044:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760776.879119:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336832 00000100:00100000:2.0:1789760776.879129:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336832 00000100:00100000:2.0:1789760776.879183:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758072106116 time 758129245624 state 0x0 ref 1 00000100:00100000:2.0:1789760776.879198:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758129506040 time 758129245624 state 0x1 ref 1 00000100:00100000:2.0:1789760776.879230:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758129506040 time 758194532578 state 0x0 ref 1 00000100:00100000:2.0:1789760776.879243:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760776.879290:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22450176, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:2.0:1789760776.879298:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.879313:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760776.879317:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.879324:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.879339:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.932556:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00002000:00010000:2.0:1789760776.932920:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:2.0:1789760776.932934:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932947:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933011:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933019:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760776.933024:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933032:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933051:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933059:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b921a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.933181:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53936us (54142us total) trans 4294972801 rc 0/0 00000100:00100000:2.0:1789760776.933199:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5511 00000100:00100000:2.0:1789760776.933211:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758129506040 time 758194532578 state 0x2 ref 0 00000100:00100000:0.0:1789760776.939344:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760776.939559:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507336960 00000100:00100000:1.0:1789760776.939570:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507336960 00000100:00100000:1.0:1789760776.939623:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758129506040 time 758194532578 state 0x0 ref 1 00000100:00100000:1.0:1789760776.939638:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758194792994 time 758194532578 state 0x1 ref 1 00000100:00100000:1.0:1789760776.939669:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758194792994 time 758254971814 state 0x0 ref 1 00000100:00100000:1.0:1789760776.939682:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760776.939762:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22454272, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:1.0:1789760776.939771:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.939787:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760776.939791:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.939798:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.939817:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.976852:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00002000:00010000:1.0:1789760776.976946:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:1.0:1789760776.976961:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.976973:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.976981:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.976989:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760776.976994:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.977003:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.977023:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.977031:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9221 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.977162:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507336960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37478us (37822us total) trans 4294972802 rc 0/0 00000100:00100000:1.0:1789760776.977172:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5512 00000100:00100000:1.0:1789760776.977182:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758194792994 time 758254971814 state 0x2 ref 0 00000100:00100000:0.0:1789760776.984499:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760776.984575:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337088 00000100:00100000:2.0:1789760776.984586:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337088 00000100:00100000:2.0:1789760776.984640:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758194792994 time 758254971814 state 0x0 ref 1 00000100:00100000:2.0:1789760776.984655:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758255232230 time 758254971814 state 0x1 ref 1 00000100:00100000:2.0:1789760776.984779:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758255232230 time 758300080212 state 0x0 ref 1 00000100:00100000:2.0:1789760776.984793:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760776.984839:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22458368, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:2.0:1789760776.984848:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.984863:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760776.984868:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.984874:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.984888:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.020168:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00002000:00010000:3.0:1789760777.020319:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:3.0:1789760777.020335:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020346:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020355:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020363:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.020368:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020383:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020403:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.020411:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9228 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.020513:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35719us (36019us total) trans 4294972803 rc 0/0 00000100:00100000:3.0:1789760777.020532:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5513 00000100:00100000:3.0:1789760777.020893:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758255232230 time 758300080212 state 0x2 ref 0 00000100:00100000:0.0:1789760777.027492:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.027787:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337216 00000100:00100000:2.0:1789760777.027797:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337216 00000100:00100000:2.0:1789760777.027966:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758255232230 time 758300080212 state 0x0 ref 1 00000100:00100000:2.0:1789760777.027982:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758300340628 time 758300080212 state 0x1 ref 1 00000100:00100000:2.0:1789760777.028017:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758300340628 time 758343320405 state 0x0 ref 1 00000100:00100000:2.0:1789760777.028031:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.028077:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22462464, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb007 00010000:00010000:2.0:1789760777.028086:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.028100:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.028105:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.028111:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.028124:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760777.028790:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760777.051690:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00002000:00010000:2.0:1789760777.051876:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22462464 -> 22466560 00002000:00010000:2.0:1789760777.051881:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:2.0:1789760777.051884:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:2.0:1789760777.051896:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051910:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051919:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051925:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.051929:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051943:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051964:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.051971:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b922f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.052060:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24028us (24574us total) trans 4294972804 rc 0/0 00000100:00100000:2.0:1789760777.052070:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5514 00000100:00100000:2.0:1789760777.052080:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758300340628 time 758343320405 state 0x2 ref 0 00000100:00100000:0.0:1789760777.059401:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.059540:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337344 00000100:00100000:1.0:1789760777.059551:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337344 00000100:00100000:1.0:1789760777.059603:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758300340628 time 758343320405 state 0x0 ref 1 00000100:00100000:1.0:1789760777.059617:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758343580821 time 758343320405 state 0x1 ref 1 00000100:00100000:1.0:1789760777.059649:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758343580821 time 758374951434 state 0x0 ref 1 00000100:00100000:1.0:1789760777.059661:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.059740:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22466560, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.059750:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.059765:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760777.059770:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.059776:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.059796:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.084492:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:2.0:1789760777.084510:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084532:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084542:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084548:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.084553:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084569:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084589:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.084597:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9236 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.084684:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507337344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25019us (25285us total) trans 4294972805 rc 0/0 00000100:00100000:2.0:1789760777.084725:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5515 00000100:00100000:2.0:1789760777.084735:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758343580821 time 758374951434 state 0x2 ref 0 00000001:00080000:3.0:1789760777.085160:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00000100:00100000:0.0:1789760777.095773:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.096450:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337472 00000100:00100000:3.0:1789760777.096460:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337472 00000100:00100000:3.0:1789760777.096512:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758343580821 time 758374951434 state 0x0 ref 1 00000100:00100000:3.0:1789760777.096546:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758375211850 time 758374951434 state 0x1 ref 1 00000100:00100000:3.0:1789760777.096582:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758375211850 time 758411884530 state 0x0 ref 1 00000100:00100000:3.0:1789760777.096594:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.096642:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22470656, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.096652:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.096667:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.096672:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.096678:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.096719:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.119723:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760777.149653:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:3.0:1789760777.149673:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149687:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149727:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149736:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.149741:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149750:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149770:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.149778:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b923d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.149868:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507337472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53270us (54100us total) trans 4294972806 rc 0/0 00000100:00100000:3.0:1789760777.149878:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5516 00000100:00100000:3.0:1789760777.149887:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758375211850 time 758411884530 state 0x2 ref 0 00000001:00080000:1.0:1789760777.150348:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00000100:00100000:1.0:1789760777.161742:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.161965:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337600 00000100:00100000:2.0:1789760777.161975:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337600 00000100:00100000:2.0:1789760777.162030:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758375211850 time 758411884530 state 0x0 ref 1 00000100:00100000:2.0:1789760777.162046:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758412144946 time 758411884530 state 0x1 ref 1 00000100:00100000:2.0:1789760777.162078:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758412144946 time 758477381584 state 0x0 ref 1 00000100:00100000:2.0:1789760777.162090:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.162134:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22474752, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.162142:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.162157:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.162161:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.162167:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.162181:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.205554:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00002000:00010000:2.0:1789760777.206252:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:2.0:1789760777.206268:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206282:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206293:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206300:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.206306:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206316:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206342:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.206350:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9244 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.206580:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44486us (44843us total) trans 4294972807 rc 0/0 00000100:00100000:2.0:1789760777.206590:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5517 00000100:00100000:2.0:1789760777.206601:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758412144946 time 758477381584 state 0x2 ref 0 00000100:00100000:2.0:1789760777.212986:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.213138:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337728 00000100:00100000:1.0:1789760777.213149:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337728 00000100:00100000:1.0:1789760777.213205:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758412144946 time 758477381584 state 0x0 ref 1 00000100:00100000:1.0:1789760777.213221:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758477642000 time 758477381584 state 0x1 ref 1 00000100:00100000:1.0:1789760777.213251:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758477642000 time 758528553811 state 0x0 ref 1 00000100:00100000:1.0:1789760777.213264:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.213312:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22478848, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.213320:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.213335:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760777.213339:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.213345:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.213359:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.237663:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00002000:00010000:2.0:1789760777.237856:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:2.0:1789760777.237927:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237937:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237943:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237948:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.237951:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237960:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.238044:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.238050:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b924b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.238126:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24859us (25145us total) trans 4294972808 rc 0/0 00000100:00100000:2.0:1789760777.238136:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5518 00000100:00100000:2.0:1789760777.238318:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758477642000 time 758528553811 state 0x2 ref 0 00000100:00100000:1.0:1789760777.242732:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.243302:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337856 00000100:00100000:3.0:1789760777.243312:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337856 00000100:00100000:3.0:1789760777.243408:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758477642000 time 758528553811 state 0x0 ref 1 00000100:00100000:3.0:1789760777.243420:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758528814227 time 758528553811 state 0x1 ref 1 00000100:00100000:3.0:1789760777.243596:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758528814227 time 758558897648 state 0x0 ref 1 00000100:00100000:3.0:1789760777.243610:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.243656:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22482944, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.243666:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.243687:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.243691:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.243719:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.243734:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.274205:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22482944 -> 22487040 00010000:00010000:3.0:1789760777.274225:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274238:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274248:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274256:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.274260:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274270:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274288:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.274295:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9252 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.274385:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507337856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30773us (31688us total) trans 4294972809 rc 0/0 00000100:00100000:3.0:1789760777.274396:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5519 00000100:00100000:3.0:1789760777.274405:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758528814227 time 758558897648 state 0x2 ref 0 00000001:00080000:3.0:1789760777.274446:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00000100:00100000:1.0:1789760777.278942:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.279048:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507337984 00000100:00100000:2.0:1789760777.279058:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507337984 00000100:00100000:2.0:1789760777.279114:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758528814227 time 758558897648 state 0x0 ref 1 00000100:00100000:2.0:1789760777.279129:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758559158064 time 758558897648 state 0x1 ref 1 00000100:00100000:2.0:1789760777.279199:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758559158064 time 758594501043 state 0x0 ref 1 00000100:00100000:2.0:1789760777.279212:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.279261:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22487040, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.279270:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.279286:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.279291:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.279297:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.279312:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.304412:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00002000:00010000:3.0:1789760777.304799:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:3.0:1789760777.304814:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304827:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304835:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304843:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.304848:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304858:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304886:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.304894:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9259 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.305019:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507337984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25805us (26082us total) trans 4294972810 rc 0/0 00000100:00100000:3.0:1789760777.305030:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5520 00000100:00100000:3.0:1789760777.305040:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758559158064 time 758594501043 state 0x2 ref 0 00000100:00100000:2.0:1789760777.309276:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.309405:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338112 00000100:00100000:1.0:1789760777.309415:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338112 00000100:00100000:1.0:1789760777.309463:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758559158064 time 758594501043 state 0x0 ref 1 00000100:00100000:1.0:1789760777.309477:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758594761459 time 758594501043 state 0x1 ref 1 00000100:00100000:1.0:1789760777.309500:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758594761459 time 758624803882 state 0x0 ref 1 00000100:00100000:1.0:1789760777.309511:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507338112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.309574:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22491136, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.309582:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.309598:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760777.309602:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.309607:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.309621:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.351333:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:2.0:1789760777.351417:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22491136 -> 22495232 00010000:00010000:2.0:1789760777.351432:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351444:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351453:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351461:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.351466:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351475:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351501:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.351509:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9260 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.351639:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507338112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42124us (42366us total) trans 4294972811 rc 0/0 00000100:00100000:2.0:1789760777.351649:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5521 00000100:00100000:2.0:1789760777.351658:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758594761459 time 758624803882 state 0x2 ref 0 00000100:00100000:3.0:1789760777.354367:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.354731:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338240 00000100:00100000:2.0:1789760777.354741:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338240 00000100:00100000:2.0:1789760777.354791:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758594761459 time 758624803882 state 0x0 ref 1 00000100:00100000:2.0:1789760777.354805:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758625064298 time 758624803882 state 0x1 ref 1 00000100:00100000:2.0:1789760777.354836:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758625064298 time 758670139825 state 0x0 ref 1 00000100:00100000:2.0:1789760777.354848:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.354891:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22495232, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.354899:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.354915:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.354921:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.354928:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.354941:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.389323:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00002000:00010000:3.0:1789760777.389486:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:3.0:1789760777.389501:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389514:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389533:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389540:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.389544:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389564:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389583:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.389591:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9267 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.389686:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34833us (35321us total) trans 4294972812 rc 0/0 00000100:00100000:3.0:1789760777.389729:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5522 00000100:00100000:3.0:1789760777.389739:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758625064298 time 758670139825 state 0x2 ref 0 00000100:00100000:0.0:1789760777.393218:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.393302:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338368 00000100:00100000:3.0:1789760777.393311:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338368 00000100:00100000:3.0:1789760777.393365:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758625064298 time 758670139825 state 0x0 ref 1 00000100:00100000:3.0:1789760777.393381:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758670400241 time 758670139825 state 0x1 ref 1 00000100:00100000:3.0:1789760777.393414:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758670400241 time 758708717310 state 0x0 ref 1 00000100:00100000:3.0:1789760777.393427:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507338368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.393483:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22499328, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.393493:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.393512:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.393517:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.393535:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.393551:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.420572:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00002000:00010000:3.0:1789760777.420784:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:3.0:1789760777.420800:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420813:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420821:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420829:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.420833:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420842:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420861:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.420868:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b926e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.420949:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507338368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27519us (27737us total) trans 4294972813 rc 0/0 00000100:00100000:3.0:1789760777.420959:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5523 00000100:00100000:3.0:1789760777.420967:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758670400241 time 758708717310 state 0x2 ref 0 00000100:00100000:2.0:1789760777.425476:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.427761:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338496 00000100:00100000:1.0:1789760777.427781:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338496 00000100:00100000:1.0:1789760777.427856:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758670400241 time 758708717310 state 0x0 ref 1 00000100:00100000:1.0:1789760777.427872:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758708977726 time 758708717310 state 0x1 ref 1 00000100:00100000:3.0:1789760777.427974:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758708977726 time 758743273157 state 0x0 ref 1 00000100:00100000:3.0:1789760777.427991:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.428041:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22503424, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.428050:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.428065:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.428070:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.428076:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.428090:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.458837:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:2.0:1789760777.459134:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:2.0:1789760777.459149:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459162:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459172:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459180:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.459185:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459194:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459213:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.459227:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9275 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.459323:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31331us (33852us total) trans 4294972814 rc 0/0 00000100:00100000:2.0:1789760777.459333:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5524 00000100:00100000:2.0:1789760777.459343:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758708977726 time 758743273157 state 0x2 ref 0 00000100:00100000:0.0:1789760777.463634:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.464110:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338624 00000100:00100000:3.0:1789760777.464122:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338624 00000100:00100000:3.0:1789760777.464188:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758708977726 time 758743273157 state 0x0 ref 1 00000100:00100000:3.0:1789760777.464202:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758743533573 time 758743273157 state 0x1 ref 1 00000100:00100000:3.0:1789760777.464232:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758743533573 time 758779536289 state 0x0 ref 1 00000100:00100000:3.0:1789760777.464244:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507338624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.464292:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22507520, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.464301:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.464317:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.464321:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.464327:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.464339:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.495223:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00002000:00010000:1.0:1789760777.495300:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22507520 -> 22511616 00010000:00010000:1.0:1789760777.495315:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495328:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495338:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495345:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760777.495350:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495360:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495380:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.495388:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b927c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.495513:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507338624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31265us (31884us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760777.495535:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5525 00000100:00100000:1.0:1789760777.495545:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758743533573 time 758779536289 state 0x2 ref 0 00000100:00100000:0.0:1789760777.501607:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.501759:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338752 00000100:00100000:3.0:1789760777.501769:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338752 00000100:00100000:3.0:1789760777.501820:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758743533573 time 758779536289 state 0x0 ref 1 00000100:00100000:3.0:1789760777.501835:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758779796705 time 758779536289 state 0x1 ref 1 00000100:00100000:3.0:1789760777.501862:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758779796705 time 758817164825 state 0x0 ref 1 00000100:00100000:3.0:1789760777.501878:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.501982:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22511616, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.501991:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.502008:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.502013:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.502020:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.502164:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.541940:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00002000:00010000:3.0:1789760777.542386:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:3.0:1789760777.542402:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542416:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542426:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542434:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.542439:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542450:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542471:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.542479:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9283 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.542590:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40710us (40988us total) trans 4294972816 rc 0/0 00000100:00100000:3.0:1789760777.542600:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5526 00000100:00100000:3.0:1789760777.542610:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758779796705 time 758817164825 state 0x2 ref 0 00000100:00100000:0.0:1789760777.550881:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.551020:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507338880 00000100:00100000:2.0:1789760777.551030:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507338880 00000100:00100000:2.0:1789760777.551095:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758779796705 time 758817164825 state 0x0 ref 1 00000100:00100000:2.0:1789760777.551111:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758817425241 time 758817164825 state 0x1 ref 1 00000100:00100000:2.0:1789760777.551144:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758817425241 time 758866446693 state 0x0 ref 1 00000100:00100000:2.0:1789760777.551157:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.551206:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22515712, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.551215:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.551230:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.551234:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.551240:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.551303:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.580964:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00002000:00010000:2.0:1789760777.581199:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:2.0:1789760777.581215:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581228:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581237:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581245:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.581249:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581258:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581281:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581289:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b928a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.581398:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507338880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30238us (30522us total) trans 4294972817 rc 0/0 00000100:00100000:2.0:1789760777.581410:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5527 00000100:00100000:2.0:1789760777.581421:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758817425241 time 758866446693 state 0x2 ref 0 00000100:00100000:0.0:1789760777.586027:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.586419:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339008 00000100:00100000:1.0:1789760777.586433:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339008 00000100:00100000:1.0:1789760777.586494:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758817425241 time 758866446693 state 0x0 ref 1 00000100:00100000:1.0:1789760777.586510:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758866707109 time 758866446693 state 0x1 ref 1 00000100:00100000:1.0:1789760777.586548:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758866707109 time 758901850981 state 0x0 ref 1 00000100:00100000:1.0:1789760777.586561:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507339008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.586621:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22519808, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.586630:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.586646:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760777.586650:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.586656:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.586669:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.608828:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:3.0:1789760777.608850:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608864:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608874:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608881:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.608888:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608897:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608917:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.608925:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9291 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.609034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00000100:00100000:3.0:1789760777.609154:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507339008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22471us (23014us total) trans 4294972818 rc 0/0 00000100:00100000:3.0:1789760777.609167:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5528 00000100:00100000:3.0:1789760777.609176:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758866707109 time 758901850981 state 0x2 ref 0 00000100:00100000:0.0:1789760777.611835:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.611942:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339136 00000100:00100000:3.0:1789760777.611956:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339136 00000100:00100000:3.0:1789760777.612016:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758866707109 time 758901850981 state 0x0 ref 1 00000100:00100000:3.0:1789760777.612031:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758902111397 time 758901850981 state 0x1 ref 1 00000100:00100000:3.0:1789760777.612062:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758902111397 time 758927365198 state 0x0 ref 1 00000100:00100000:3.0:1789760777.612074:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.612215:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22523904, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.612223:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.612236:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760777.612239:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.612251:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.612262:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.647433:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:3.0:1789760777.647452:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647465:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647474:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647481:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.647486:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647495:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647539:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.647547:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9298 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.647661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00000100:00100000:3.0:1789760777.647853:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35775us (36018us total) trans 4294972819 rc 0/0 00000100:00100000:3.0:1789760777.647864:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5529 00000100:00100000:3.0:1789760777.647874:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758902111397 time 758927365198 state 0x2 ref 0 00000100:00100000:0.0:1789760777.652272:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.652361:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339264 00000100:00100000:2.0:1789760777.652372:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339264 00000100:00100000:2.0:1789760777.652425:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758902111397 time 758927365198 state 0x0 ref 1 00000100:00100000:2.0:1789760777.652440:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758927625614 time 758927365198 state 0x1 ref 1 00000100:00100000:2.0:1789760777.652529:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758927625614 time 758967810320 state 0x0 ref 1 00000100:00100000:2.0:1789760777.652542:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507339264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.652844:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22528000, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.652854:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.652870:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760777.652875:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.652881:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.652899:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.682412:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00002000:00010000:2.0:1789760777.682531:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:2.0:1789760777.682546:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682629:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682638:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682646:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.682651:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682660:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682684:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.682712:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b929f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.682853:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507339264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30308us (30586us total) trans 4294972820 rc 0/0 00000100:00100000:2.0:1789760777.682864:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5530 00000100:00100000:2.0:1789760777.682873:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758927625614 time 758967810320 state 0x2 ref 0 00000100:00100000:0.0:1789760777.693227:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.693328:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339392 00000100:00100000:3.0:1789760777.693337:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339392 00000100:00100000:3.0:1789760777.693389:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758927625614 time 758967810320 state 0x0 ref 1 00000100:00100000:3.0:1789760777.693405:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758968070736 time 758967810320 state 0x1 ref 1 00000100:00100000:3.0:1789760777.693478:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758968070736 time 759008780455 state 0x0 ref 1 00000100:00100000:3.0:1789760777.693492:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507339392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.693550:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22532096, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.693559:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.693738:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760777.693743:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.693748:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.693763:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.720176:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22532096 -> 22536192 00010000:00010000:3.0:1789760777.720198:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720212:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720222:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720230:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.720234:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720244:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720264:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.720274:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.720408:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507339392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26913us (27186us total) trans 4294972821 rc 0/0 00000100:00100000:3.0:1789760777.720419:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5531 00000100:00100000:3.0:1789760777.720429:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758968070736 time 759008780455 state 0x2 ref 0 00000001:00080000:3.0:1789760777.720467:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00000100:00100000:0.0:1789760777.727948:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.728094:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339520 00000100:00100000:3.0:1789760777.728104:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339520 00000100:00100000:3.0:1789760777.728164:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 758968070736 time 759008780455 state 0x0 ref 1 00000100:00100000:3.0:1789760777.728178:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759009040871 time 759008780455 state 0x1 ref 1 00000100:00100000:3.0:1789760777.728246:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759009040871 time 759043547179 state 0x0 ref 1 00000100:00100000:3.0:1789760777.728260:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507339520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.728309:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22536192, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.728318:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.728335:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760777.728340:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.728346:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.728361:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.764073:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00002000:00010000:3.0:1789760777.764139:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:3.0:1789760777.764160:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764346:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764357:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764366:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760777.764371:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764381:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764408:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.764416:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.764632:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507339520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36366us (36684us total) trans 4294972822 rc 0/0 00000100:00100000:3.0:1789760777.764644:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5532 00000100:00100000:3.0:1789760777.764656:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759009040871 time 759043547179 state 0x2 ref 0 00000100:00100000:0.0:1789760777.771858:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760777.773471:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339648 00000100:00100000:2.0:1789760777.773490:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339648 00000100:00100000:2.0:1789760777.773837:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759009040871 time 759043547179 state 0x0 ref 1 00000100:00100000:2.0:1789760777.773858:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759043807595 time 759043547179 state 0x1 ref 1 00000100:00100000:2.0:1789760777.773902:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759043807595 time 759089203998 state 0x0 ref 1 00000100:00100000:2.0:1789760777.773920:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760777.774003:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22540288, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:2.0:1789760777.774015:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.774038:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760777.774043:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.774052:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.774071:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.826892:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:2.0:1789760777.827238:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22540288 -> 22544384 00010000:00010000:2.0:1789760777.827260:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827368:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827380:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827388:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760777.827394:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827403:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827428:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.827437:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b92b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.827687:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53765us (55837us total) trans 4294972823 rc 0/0 00000100:00100000:2.0:1789760777.827893:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5533 00000100:00100000:2.0:1789760777.827905:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759043807595 time 759089203998 state 0x2 ref 0 00000100:00100000:0.0:1789760777.831254:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.833027:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339776 00000100:00100000:3.0:1789760777.833046:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339776 00000100:00100000:3.0:1789760777.833136:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759043807595 time 759089203998 state 0x0 ref 1 00000100:00100000:3.0:1789760777.833155:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759089464414 time 759089203998 state 0x1 ref 1 00000100:00100000:3.0:1789760777.833348:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759089464414 time 759148649143 state 0x0 ref 1 00000100:00100000:3.0:1789760777.833366:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507339776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.833437:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22544384, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.833448:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.833468:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760777.833474:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.833482:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.833499:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760777.880047:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:1.0:1789760777.880073:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880088:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880099:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880106:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760777.880111:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880121:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880147:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.880156:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b92bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.880284:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507339776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46915us (49038us total) trans 4294972824 rc 0/0 00000100:00100000:1.0:1789760777.880297:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5534 00000100:00100000:1.0:1789760777.880309:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759089464414 time 759148649143 state 0x2 ref 0 00000001:00080000:0.0:1789760777.881070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00000100:00100000:0.0:1789760777.885990:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760777.886132:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507339904 00000100:00100000:3.0:1789760777.886145:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507339904 00000100:00100000:3.0:1789760777.886340:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759089464414 time 759148649143 state 0x0 ref 1 00000100:00100000:3.0:1789760777.886360:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759148909559 time 759148649143 state 0x1 ref 1 00000100:00100000:3.0:1789760777.886437:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759148909559 time 759201739189 state 0x0 ref 1 00000100:00100000:3.0:1789760777.886451:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760777.886531:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22548480, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:3.0:1789760777.886543:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.886561:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760777.886567:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.886576:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.886592:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.940342:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00002000:00010000:1.0:1789760777.940511:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22548480 -> 22552576 00010000:00010000:1.0:1789760777.940546:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940560:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940569:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940577:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760777.940582:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940592:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940616:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940624:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b92c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.941060:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507339904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54603us (55079us total) trans 4294972825 rc 0/0 00000100:00100000:1.0:1789760777.941075:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5535 00000100:00100000:1.0:1789760777.941087:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759148909559 time 759201739189 state 0x2 ref 0 00000100:00100000:2.0:1789760777.947422:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760777.947676:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340032 00000100:00100000:0.0:1789760777.948161:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340032 00000100:00100000:1.0:1789760777.949064:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759148909559 time 759201739189 state 0x0 ref 1 00000100:00100000:1.0:1789760777.949086:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759201999605 time 759201739189 state 0x1 ref 1 00000100:00100000:1.0:1789760777.949131:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759201999605 time 759264434016 state 0x0 ref 1 00000100:00100000:1.0:1789760777.949149:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507340032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.949590:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22552576, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.949603:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.949622:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760777.949627:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.949634:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.949651:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.989312:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:1.0:1789760777.989666:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:1.0:1789760777.989686:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989740:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989757:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989766:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760777.989771:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989781:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989807:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.989816:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.989932:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507340032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40782us (42521us total) trans 4294972826 rc 0/0 00000100:00100000:1.0:1789760777.989944:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5536 00000100:00100000:1.0:1789760777.990208:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759201999605 time 759264434016 state 0x2 ref 0 00000100:00100000:2.0:1789760777.992561:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760777.992910:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340160 00000100:00100000:1.0:1789760777.992924:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340160 00000100:00100000:1.0:1789760777.993001:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759201999605 time 759264434016 state 0x0 ref 1 00000100:00100000:1.0:1789760777.993021:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759264694432 time 759264434016 state 0x1 ref 1 00000100:00100000:1.0:1789760777.993114:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759264694432 time 759308415240 state 0x0 ref 1 00000100:00100000:1.0:1789760777.993130:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507340160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760777.993202:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22556672, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb007 00010000:00010000:1.0:1789760777.993223:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.993244:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760777.993250:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.993257:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.993275:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.028559:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00002000:00010000:1.0:1789760778.029069:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22556672 -> 22560768 00002000:00010000:1.0:1789760778.029075:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:1.0:1789760778.029078:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00010000:00010000:1.0:1789760778.029104:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029121:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029132:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029141:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.029147:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029158:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029184:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.029192:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.029316:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507340160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36182us (36770us total) trans 4294972827 rc 0/0 00000100:00100000:1.0:1789760778.029327:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5537 00000100:00100000:1.0:1789760778.029339:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759264694432 time 759308415240 state 0x2 ref 0 00000100:00100000:0.0:1789760778.034093:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.034201:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340288 00000100:00100000:1.0:1789760778.034216:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340288 00000100:00100000:1.0:1789760778.034284:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759264694432 time 759308415240 state 0x0 ref 1 00000100:00100000:1.0:1789760778.034303:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759308675656 time 759308415240 state 0x1 ref 1 00000100:00100000:1.0:1789760778.034379:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759308675656 time 759349682073 state 0x0 ref 1 00000100:00100000:1.0:1789760778.034398:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507340288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.034464:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22560768, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.034475:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.034496:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.034501:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.034508:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.034661:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.070281:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00002000:00010000:2.0:1789760778.070450:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:2.0:1789760778.070470:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070484:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070495:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070502:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760778.070508:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070530:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070564:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.070580:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.070903:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507340288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36500us (36820us total) trans 4294972828 rc 0/0 00000100:00100000:2.0:1789760778.070917:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5538 00000100:00100000:2.0:1789760778.070929:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759308675656 time 759349682073 state 0x2 ref 0 00000100:00100000:0.0:1789760778.077945:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.078032:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340416 00000100:00100000:1.0:1789760778.078045:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340416 00000100:00100000:1.0:1789760778.078137:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759308675656 time 759349682073 state 0x0 ref 1 00000100:00100000:1.0:1789760778.078156:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759349942489 time 759349682073 state 0x1 ref 1 00000100:00100000:1.0:1789760778.078193:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759349942489 time 759393495488 state 0x0 ref 1 00000100:00100000:1.0:1789760778.078209:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507340416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.078287:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22564864, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.078298:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.078329:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.078334:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.078342:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.078359:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.119046:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00002000:00010000:0.0:1789760778.119136:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:0.0:1789760778.119155:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119169:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119178:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119186:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760778.119192:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119207:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119231:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.119240:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b92de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.119361:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507340416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41149us (41424us total) trans 4294972829 rc 0/0 00000100:00100000:0.0:1789760778.119373:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5539 00000100:00100000:0.0:1789760778.119385:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759349942489 time 759393495488 state 0x2 ref 0 00000100:00100000:0.0:1789760778.123207:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.123314:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340544 00000100:00100000:1.0:1789760778.123329:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340544 00000100:00100000:1.0:1789760778.123408:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759349942489 time 759393495488 state 0x0 ref 1 00000100:00100000:1.0:1789760778.123429:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759393755904 time 759393495488 state 0x1 ref 1 00000100:00100000:1.0:1789760778.123463:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759393755904 time 759438766019 state 0x0 ref 1 00000100:00100000:1.0:1789760778.123475:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507340544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.123555:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22568960, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.123567:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.123587:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.123592:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.123600:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.123621:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.144424:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760778.155257:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:1.0:1789760778.155285:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155300:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155310:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155319:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.155325:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155335:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155360:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.155370:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.155504:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507340544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32023us (32305us total) trans 4294972830 rc 0/0 00000100:00100000:1.0:1789760778.155955:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5540 00000100:00100000:1.0:1789760778.155977:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759393755904 time 759438766019 state 0x2 ref 0 00000001:00080000:3.0:1789760778.156050:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00000100:00100000:3.0:1789760778.160843:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760778.161007:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340672 00000100:00100000:2.0:1789760778.161088:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340672 00000100:00100000:2.0:1789760778.161316:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759393755904 time 759438766019 state 0x0 ref 1 00000100:00100000:2.0:1789760778.161336:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759439026435 time 759438766019 state 0x1 ref 1 00000100:00100000:2.0:1789760778.161402:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759439026435 time 759476704467 state 0x0 ref 1 00000100:00100000:2.0:1789760778.161425:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507340672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760778.161492:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22573056, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:2.0:1789760778.161504:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.161535:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760778.161540:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.161549:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.161573:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.220085:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:3.0:1789760778.220177:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:3.0:1789760778.220200:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220217:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220228:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220236:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.220242:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220253:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220283:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.220291:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b92ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.220420:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507340672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58997us (59821us total) trans 4294972831 rc 0/0 00000100:00100000:3.0:1789760778.220434:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5541 00000100:00100000:3.0:1789760778.220447:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759439026435 time 759476704467 state 0x2 ref 0 00000100:00100000:0.0:1789760778.226384:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.226494:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340800 00000100:00100000:1.0:1789760778.226509:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340800 00000100:00100000:1.0:1789760778.226596:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759439026435 time 759476704467 state 0x0 ref 1 00000100:00100000:1.0:1789760778.226615:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759476964883 time 759476704467 state 0x1 ref 1 00000100:00100000:1.0:1789760778.226683:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759476964883 time 759541984870 state 0x0 ref 1 00000100:00100000:1.0:1789760778.226734:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507340800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.226799:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22577152, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.226810:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.226829:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.226834:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.226841:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.226858:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.286628:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22577152 -> 22581248 00010000:00010000:2.0:1789760778.286660:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286678:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286689:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286823:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760778.286829:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286841:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286868:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.286877:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b92f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.287011:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507340800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60309us (60637us total) trans 4294972832 rc 0/0 00000100:00100000:2.0:1789760778.287461:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5542 00000100:00100000:2.0:1789760778.287476:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759476964883 time 759541984870 state 0x2 ref 0 00000001:00080000:3.0:1789760778.287831:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00000100:00100000:0.0:1789760778.291378:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.291452:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507340928 00000100:00100000:1.0:1789760778.291465:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507340928 00000100:00100000:1.0:1789760778.291552:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759476964883 time 759541984870 state 0x0 ref 1 00000100:00100000:1.0:1789760778.291570:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759542245286 time 759541984870 state 0x1 ref 1 00000100:00100000:1.0:1789760778.291605:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759542245286 time 759606908169 state 0x0 ref 1 00000100:00100000:1.0:1789760778.291621:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507340928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.291685:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22581248, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.291735:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.291755:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.291760:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.291768:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.291787:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.330098:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22581248 -> 22585344 00010000:00010000:3.0:1789760778.330129:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330146:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330156:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330165:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.330170:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330181:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330206:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.330215:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b92fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.330349:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507340928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38725us (38974us total) trans 4294972833 rc 0/0 00000100:00100000:3.0:1789760778.330365:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5543 00000100:00100000:3.0:1789760778.330377:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759542245286 time 759606908169 state 0x2 ref 0 00000001:00080000:0.0:1789760778.331150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00000100:00100000:0.0:1789760778.333916:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.334222:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341056 00000100:00100000:3.0:1789760778.334235:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341056 00000100:00100000:3.0:1789760778.334305:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759542245286 time 759606908169 state 0x0 ref 1 00000100:00100000:3.0:1789760778.334323:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759607168585 time 759606908169 state 0x1 ref 1 00000100:00100000:3.0:1789760778.334390:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759607168585 time 759649692628 state 0x0 ref 1 00000100:00100000:3.0:1789760778.334404:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507341056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.334466:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22585344, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.334476:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.334494:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.334499:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.334507:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.334533:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.357010:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00002000:00010000:3.0:1789760778.357087:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:3.0:1789760778.357107:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357124:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357136:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357145:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.357151:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357162:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357187:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.357193:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9301 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.357410:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507341056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22997us (23497us total) trans 4294972834 rc 0/0 00000100:00100000:3.0:1789760778.357420:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5544 00000100:00100000:3.0:1789760778.357431:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759607168585 time 759649692628 state 0x2 ref 0 00000100:00100000:2.0:1789760778.360191:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.360428:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341184 00000100:00100000:1.0:1789760778.360441:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341184 00000100:00100000:1.0:1789760778.360526:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759607168585 time 759649692628 state 0x0 ref 1 00000100:00100000:1.0:1789760778.360546:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759649953044 time 759649692628 state 0x1 ref 1 00000100:00100000:1.0:1789760778.360584:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759649953044 time 759675886868 state 0x0 ref 1 00000100:00100000:1.0:1789760778.360604:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.360666:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22589440, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.360677:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.360739:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760778.360746:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.360754:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.360773:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.395683:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00002000:00010000:3.0:1789760778.396038:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22589440 -> 22593536 00010000:00010000:3.0:1789760778.396059:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396074:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396084:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396092:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.396098:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396109:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396134:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.396141:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9308 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.396308:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35702us (36125us total) trans 4294972835 rc 0/0 00000100:00100000:3.0:1789760778.396321:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5545 00000100:00100000:3.0:1789760778.396334:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759649953044 time 759675886868 state 0x2 ref 0 00000100:00100000:0.0:1789760778.399436:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.399854:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341312 00000100:00100000:3.0:1789760778.399867:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341312 00000100:00100000:3.0:1789760778.399945:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759649953044 time 759675886868 state 0x0 ref 1 00000100:00100000:3.0:1789760778.400027:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759676147284 time 759675886868 state 0x1 ref 1 00000100:00100000:3.0:1789760778.400094:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759676147284 time 759715396540 state 0x0 ref 1 00000100:00100000:3.0:1789760778.400109:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507341312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.400175:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22593536, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.400186:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.400205:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.400211:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.400219:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.400235:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.442069:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:1.0:1789760778.442210:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22593536 -> 22597632 00010000:00010000:1.0:1789760778.442233:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442249:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442260:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442272:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.442278:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442289:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442313:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.442322:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b930f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.442485:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507341312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42373us (43056us total) trans 4294972836 rc 0/0 00000100:00100000:1.0:1789760778.442499:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5546 00000100:00100000:1.0:1789760778.442521:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759676147284 time 759715396540 state 0x2 ref 0 00000100:00100000:2.0:1789760778.445904:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.446044:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341440 00000100:00100000:3.0:1789760778.446058:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341440 00000100:00100000:3.0:1789760778.446135:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759676147284 time 759715396540 state 0x0 ref 1 00000100:00100000:3.0:1789760778.446155:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759715656956 time 759715396540 state 0x1 ref 1 00000100:00100000:3.0:1789760778.446239:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759715656956 time 759761540998 state 0x0 ref 1 00000100:00100000:3.0:1789760778.446254:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507341440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.446326:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22597632, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.446337:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.446358:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.446364:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.446372:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.446391:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.481826:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00002000:00010000:3.0:1789760778.481951:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:3.0:1789760778.482029:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482045:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482055:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482064:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.482071:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482081:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482105:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.482113:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9316 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.482251:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507341440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35993us (36358us total) trans 4294972837 rc 0/0 00000100:00100000:3.0:1789760778.482263:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5547 00000100:00100000:3.0:1789760778.482275:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759715656956 time 759761540998 state 0x2 ref 0 00000100:00100000:2.0:1789760778.486136:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.486240:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341568 00000100:00100000:3.0:1789760778.486252:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341568 00000100:00100000:3.0:1789760778.486322:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759715656956 time 759761540998 state 0x0 ref 1 00000100:00100000:3.0:1789760778.486342:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759761801414 time 759761540998 state 0x1 ref 1 00000100:00100000:3.0:1789760778.486415:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759761801414 time 759801717608 state 0x0 ref 1 00000100:00100000:3.0:1789760778.486429:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.486492:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22601728, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.486504:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.486534:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.486540:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.486554:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.486573:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.529767:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00002000:00010000:1.0:1789760778.531953:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:1.0:1789760778.532053:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532070:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532088:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532097:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.532103:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532114:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532140:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.532150:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b931d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.532360:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45924us (46229us total) trans 4294972838 rc 0/0 00000100:00100000:1.0:1789760778.532373:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5548 00000100:00100000:1.0:1789760778.532386:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759761801414 time 759801717608 state 0x2 ref 0 00000100:00100000:1.0:1789760778.536241:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.536351:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341696 00000100:00100000:3.0:1789760778.536365:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341696 00000100:00100000:3.0:1789760778.536437:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759761801414 time 759801717608 state 0x0 ref 1 00000100:00100000:3.0:1789760778.536454:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759801978024 time 759801717608 state 0x1 ref 1 00000100:00100000:3.0:1789760778.536677:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759801978024 time 759851978497 state 0x0 ref 1 00000100:00100000:3.0:1789760778.536726:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.536793:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22605824, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.536804:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.536826:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.536832:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.536840:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.536858:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.594367:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00002000:00010000:3.0:1789760778.594495:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:3.0:1789760778.594526:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594542:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594561:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594570:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.594576:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594588:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594615:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.594623:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9324 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.594807:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58108us (58574us total) trans 4294972839 rc 0/0 00000100:00100000:3.0:1789760778.594820:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5549 00000100:00100000:3.0:1789760778.594832:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759801978024 time 759851978497 state 0x2 ref 0 00000100:00100000:3.0:1789760778.597922:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760778.598055:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341824 00000100:00100000:0.0:1789760778.598067:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341824 00000100:00100000:0.0:1789760778.598138:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759801978024 time 759851978497 state 0x0 ref 1 00000100:00100000:0.0:1789760778.598159:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759852238913 time 759851978497 state 0x1 ref 1 00000100:00100000:0.0:1789760778.598198:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759852238913 time 759913500725 state 0x0 ref 1 00000100:00100000:0.0:1789760778.598213:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760778.598278:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22609920, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:0.0:1789760778.598290:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598312:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760778.598317:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.598325:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598343:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.658749:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:1.0:1789760778.658769:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658782:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658790:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658797:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.658802:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658810:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658830:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.658839:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b932b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.658945:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507341824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 60730us (61032us total) trans 4294972840 rc 0/0 00000100:00100000:1.0:1789760778.658956:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5550 00000100:00100000:1.0:1789760778.658966:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759852238913 time 759913500725 state 0x2 ref 0 00000001:00080000:2.0:1789760778.659676:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00000100:00100000:1.0:1789760778.677502:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.677606:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507341952 00000100:00100000:3.0:1789760778.677616:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507341952 00000100:00100000:3.0:1789760778.677670:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759852238913 time 759913500725 state 0x0 ref 1 00000100:00100000:3.0:1789760778.677685:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759913761141 time 759913500725 state 0x1 ref 1 00000100:00100000:3.0:1789760778.677745:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759913761141 time 759993048116 state 0x0 ref 1 00000100:00100000:3.0:1789760778.677769:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.677823:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22614016, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.677833:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.677847:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760778.677852:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.677869:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.677884:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.729185:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:3.0:1789760778.729403:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22614016 -> 22618112 00010000:00010000:3.0:1789760778.729429:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729441:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729450:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729458:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.729463:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729473:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729500:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.729518:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9332 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.729731:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507341952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51921us (52196us total) trans 4294972841 rc 0/0 00000100:00100000:3.0:1789760778.729748:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5551 00000100:00100000:3.0:1789760778.729758:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759913761141 time 759993048116 state 0x2 ref 0 00000100:00100000:3.0:1789760778.735671:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.735867:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342080 00000100:00100000:1.0:1789760778.735877:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342080 00000100:00100000:1.0:1789760778.735995:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759913761141 time 759993048116 state 0x0 ref 1 00000100:00100000:1.0:1789760778.736008:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759993308532 time 759993048116 state 0x1 ref 1 00000100:00100000:1.0:1789760778.736025:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759993308532 time 760051330299 state 0x0 ref 1 00000100:00100000:1.0:1789760778.736035:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507342080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.736077:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22618112, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.736084:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.736097:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760778.736101:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.736105:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.736121:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.768580:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00002000:00010000:1.0:1789760778.768779:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:1.0:1789760778.768794:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768805:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768815:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768822:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.768827:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768836:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768855:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.768863:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9339 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.769026:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507342080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32987us (33360us total) trans 4294972842 rc 0/0 00000100:00100000:1.0:1789760778.769037:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5552 00000100:00100000:1.0:1789760778.769046:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759993308532 time 760051330299 state 0x2 ref 0 00000100:00100000:1.0:1789760778.780913:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760778.781008:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342208 00000100:00100000:0.0:1789760778.781019:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342208 00000100:00100000:0.0:1789760778.781070:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 759993308532 time 760051330299 state 0x0 ref 1 00000100:00100000:0.0:1789760778.781084:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760051590715 time 760051330299 state 0x1 ref 1 00000100:00100000:0.0:1789760778.781119:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760051590715 time 760096422595 state 0x0 ref 1 00000100:00100000:0.0:1789760778.781134:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760778.781177:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22622208, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:0.0:1789760778.781186:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.781281:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760778.781344:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.781352:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.781366:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.844329:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00002000:00010000:1.0:1789760778.844608:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:1.0:1789760778.844624:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844642:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844650:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844657:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.844662:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844670:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844689:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.844777:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9340 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.844951:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63815us (64044us total) trans 4294972843 rc 0/0 00000100:00100000:1.0:1789760778.844962:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5553 00000100:00100000:1.0:1789760778.844971:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760051590715 time 760096422595 state 0x2 ref 0 00000100:00100000:0.0:1789760778.849032:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.849364:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342336 00000100:00100000:3.0:1789760778.849382:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342336 00000100:00100000:3.0:1789760778.849463:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760051590715 time 760096422595 state 0x0 ref 1 00000100:00100000:3.0:1789760778.849484:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760096683011 time 760096422595 state 0x1 ref 1 00000100:00100000:3.0:1789760778.849538:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760096683011 time 760164840201 state 0x0 ref 1 00000100:00100000:3.0:1789760778.849933:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507342336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.850348:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22626304, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.850361:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.850382:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760778.850387:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.850395:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.850412:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.891975:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22626304 -> 22630400 00010000:00010000:2.0:1789760778.892003:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892019:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892030:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892039:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760778.892044:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892054:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892087:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.892096:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9347 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.892226:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507342336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42293us (43196us total) trans 4294972844 rc 0/0 00000100:00100000:2.0:1789760778.892239:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5554 00000100:00100000:2.0:1789760778.892251:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760096683011 time 760164840201 state 0x2 ref 0 00000001:00080000:2.0:1789760778.892805:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00000100:00100000:0.0:1789760778.897037:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.897160:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342464 00000100:00100000:1.0:1789760778.897183:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342464 00000100:00100000:1.0:1789760778.897263:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760096683011 time 760164840201 state 0x0 ref 1 00000100:00100000:1.0:1789760778.897283:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760165100617 time 760164840201 state 0x1 ref 1 00000100:00100000:1.0:1789760778.897317:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760165100617 time 760212620119 state 0x0 ref 1 00000100:00100000:1.0:1789760778.897332:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.897402:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22630400, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.897413:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.897441:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760778.897450:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.897458:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.897476:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.935399:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00002000:00010000:3.0:1789760778.935449:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:3.0:1789760778.935468:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935481:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935489:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935494:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760778.935498:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935505:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935539:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.935546:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b934e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.935680:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38343us (38650us total) trans 4294972845 rc 0/0 00000100:00100000:3.0:1789760778.935742:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5555 00000100:00100000:3.0:1789760778.935757:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760165100617 time 760212620119 state 0x2 ref 0 00000100:00100000:0.0:1789760778.943131:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.943230:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342592 00000100:00100000:1.0:1789760778.943312:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342592 00000100:00100000:1.0:1789760778.943385:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760165100617 time 760212620119 state 0x0 ref 1 00000100:00100000:1.0:1789760778.943403:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760212880535 time 760212620119 state 0x1 ref 1 00000100:00100000:1.0:1789760778.943441:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760212880535 time 760258743925 state 0x0 ref 1 00000100:00100000:1.0:1789760778.943460:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507342592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760778.944277:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22634496, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:1.0:1789760778.944293:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.944313:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760778.944318:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.944325:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.944342:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.969451:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.969488:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.969611:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760778.969655:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342976 00000100:00100000:1.0:1789760778.969669:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342976 00000100:00100000:1.0:1789760778.969766:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343232 00000100:00100000:1.0:1789760778.969770:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343232 00000100:00100000:1.0:1789760778.969802:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 958 00000100:00100000:1.0:1789760778.969814:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342976:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760778.969931:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342720 00000100:00100000:2.0:1789760778.969937:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342720 00000100:00100000:2.0:1789760778.969974:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 482 00000100:00100000:2.0:1789760778.969983:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507342720:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760778.970026:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507342720:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 45us (596us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.970034:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 482 00000100:00080000:2.0:1789760778.970069:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760778.970299:0:14732:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 959 00000100:00100000:0.0:1789760778.970311:0:14732:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507343232:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760778.970354:0:14732:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507343232:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 46us (743us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760778.970361:0:14732:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 959 00000100:00100000:1.0:1789760778.970591:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108040700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507342976:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 778us (1100us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.970599:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 958 00000100:00100000:2.0:1789760778.972334:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760778.972377:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760778.972410:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507342848 00000100:00100000:2.0:1789760778.972419:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507342848 00000100:00100000:2.0:1789760778.972478:0:14700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343104 00000100:00100000:2.0:1789760778.972482:0:14700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343104 00000100:00100000:2.0:1789760778.972528:0:14700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 960 00000100:00100000:2.0:1789760778.972538:0:14700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507342848:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760778.972580:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 961 00000100:00100000:0.0:1789760778.972592:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507343104:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:0.0:1789760778.972673:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507343104:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 84us (295us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760778.972683:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 961 00000100:00100000:2.0:1789760778.972974:0:14700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507342848:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 436us (649us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.972981:0:14700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 960 00000100:00080000:2.0:1789760778.973005:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:1.0:1789760778.988870:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:1.0:1789760778.988896:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988911:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988921:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988929:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760778.988936:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988947:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988972:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.988980:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9355 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.989105:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507342592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45639us (45978us total) trans 4294972846 rc 0/0 00000100:00100000:1.0:1789760778.989117:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5556 00000100:00100000:1.0:1789760778.989130:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760212880535 time 760258743925 state 0x2 ref 0 00000001:00080000:3.0:1789760778.989224:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00000100:00100000:2.0:1789760778.998065:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760778.999163:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343360 00000100:00100000:3.0:1789760778.999181:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343360 00000100:00100000:3.0:1789760778.999276:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760212880535 time 760258743925 state 0x0 ref 1 00000100:00100000:3.0:1789760778.999295:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760259004341 time 760258743925 state 0x1 ref 1 00000100:00100000:3.0:1789760778.999336:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760259004341 time 760314639406 state 0x0 ref 1 00000100:00100000:3.0:1789760778.999351:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507343360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760778.999425:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22638592, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb007 00010000:00010000:3.0:1789760778.999449:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.999469:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760778.999474:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.999480:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.999498:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.045851:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00002000:00010000:2.0:1789760779.046056:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22638592 -> 22642688 00002000:00010000:2.0:1789760779.046062:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:2.0:1789760779.046065:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:2.0:1789760779.046082:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046097:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046107:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046116:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.046122:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046132:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046163:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.046173:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b935c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.046312:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507343360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46957us (48256us total) trans 4294972847 rc 0/0 00000100:00100000:2.0:1789760779.046325:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5557 00000100:00100000:2.0:1789760779.046337:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760259004341 time 760314639406 state 0x2 ref 0 00000100:00100000:2.0:1789760779.052619:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760779.052990:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343488 00000100:00100000:3.0:1789760779.053002:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343488 00000100:00100000:2.0:1789760779.053224:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760259004341 time 760314639406 state 0x0 ref 1 00000100:00100000:2.0:1789760779.053361:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760314899822 time 760314639406 state 0x1 ref 1 00000100:00100000:2.0:1789760779.053402:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760314899822 time 760368704101 state 0x0 ref 1 00000100:00100000:2.0:1789760779.053416:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507343488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.053482:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22642688, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.053493:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.053522:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760779.053528:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.053536:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.053553:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760779.054441:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760779.093926:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:2.0:1789760779.094109:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:2.0:1789760779.094131:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094146:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094157:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094166:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.094172:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094183:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094208:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.094222:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9363 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.094385:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507343488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40965us (41773us total) trans 4294972848 rc 0/0 00000100:00100000:2.0:1789760779.094398:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5558 00000100:00100000:2.0:1789760779.094410:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760314899822 time 760368704101 state 0x2 ref 0 00000100:00100000:3.0:1789760779.106051:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.106325:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343616 00000100:00100000:1.0:1789760779.106338:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343616 00000100:00100000:1.0:1789760779.106413:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760314899822 time 760368704101 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106432:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760368964517 time 760368704101 state 0x1 ref 1 00000100:00100000:1.0:1789760779.106471:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760368964517 time 760421773234 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106490:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507343616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.106563:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22646784, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.106573:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106602:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.106608:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.106616:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106633:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.146138:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:2.0:1789760779.146164:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146179:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146189:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146197:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.146202:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146212:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146243:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.146251:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b936a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.146378:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507343616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39887us (40336us total) trans 4294972849 rc 0/0 00000100:00100000:2.0:1789760779.146390:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5559 00000100:00100000:2.0:1789760779.146402:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760368964517 time 760421773234 state 0x2 ref 0 00000001:00080000:0.0:1789760779.146490:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00000100:00100000:1.0:1789760779.151193:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.151295:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343744 00000100:00100000:2.0:1789760779.151310:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343744 00000100:00100000:2.0:1789760779.151387:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760368964517 time 760421773234 state 0x0 ref 1 00000100:00100000:2.0:1789760779.151406:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760422033650 time 760421773234 state 0x1 ref 1 00000100:00100000:2.0:1789760779.151477:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760422033650 time 760466779150 state 0x0 ref 1 00000100:00100000:2.0:1789760779.151493:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507343744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.151584:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22650880, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.151596:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151615:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760779.151622:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.151629:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151650:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.163870:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760779.178076:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:3.0:1789760779.178202:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:3.0:1789760779.178222:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178238:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178249:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178258:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760779.178263:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178274:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178297:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.178306:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9371 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.178426:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507343744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26929us (27241us total) trans 4294972850 rc 0/0 00000100:00100000:3.0:1789760779.178439:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5560 00000100:00100000:3.0:1789760779.178451:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760422033650 time 760466779150 state 0x2 ref 0 00000100:00100000:3.0:1789760779.181904:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.182399:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507343872 00000100:00100000:2.0:1789760779.182413:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507343872 00000100:00100000:2.0:1789760779.182492:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760422033650 time 760466779150 state 0x0 ref 1 00000100:00100000:2.0:1789760779.182521:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760467039566 time 760466779150 state 0x1 ref 1 00000100:00100000:2.0:1789760779.182667:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760467039566 time 760497968773 state 0x0 ref 1 00000100:00100000:2.0:1789760779.182684:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507343872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.182893:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22654976, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.182905:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.182925:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760779.182930:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.182937:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.182953:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.224478:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:0.0:1789760779.229686:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:0.0:1789760779.229755:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229771:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229780:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229788:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.229793:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229813:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229836:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.229843:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9378 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.229963:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507343872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47277us (48069us total) trans 4294972851 rc 0/0 00000100:00100000:0.0:1789760779.229973:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5561 00000100:00100000:0.0:1789760779.229983:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760467039566 time 760497968773 state 0x2 ref 0 00000100:00100000:2.0:1789760779.234063:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.234304:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344000 00000100:00100000:1.0:1789760779.234333:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344000 00000100:00100000:1.0:1789760779.234392:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760467039566 time 760497968773 state 0x0 ref 1 00000100:00100000:1.0:1789760779.234411:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760498229189 time 760497968773 state 0x1 ref 1 00000100:00100000:1.0:1789760779.234442:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760498229189 time 760549745177 state 0x0 ref 1 00000100:00100000:1.0:1789760779.234454:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507344000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.234502:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22659072, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.234521:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.234537:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.234551:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.234557:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.234573:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.266361:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:1.0:1789760779.266434:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22659072 -> 22663168 00010000:00010000:1.0:1789760779.266449:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266461:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266470:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266478:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.266482:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266491:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266521:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266530:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b937f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.266849:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507344000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32391us (32794us total) trans 4294972852 rc 0/0 00000100:00100000:1.0:1789760779.266859:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5562 00000100:00100000:1.0:1789760779.266869:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760498229189 time 760549745177 state 0x2 ref 0 00000100:00100000:1.0:1789760779.271265:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760779.271660:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344128 00000100:00100000:3.0:1789760779.271671:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344128 00000100:00100000:3.0:1789760779.271778:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760498229189 time 760549745177 state 0x0 ref 1 00000100:00100000:3.0:1789760779.271793:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760550005593 time 760549745177 state 0x1 ref 1 00000100:00100000:3.0:1789760779.271814:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760550005593 time 760587117232 state 0x0 ref 1 00000100:00100000:3.0:1789760779.271827:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760779.271876:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22663168, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:3.0:1789760779.271885:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.271902:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760779.271906:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.271912:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.271927:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.295396:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00002000:00010000:3.0:1789760779.295685:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:3.0:1789760779.295768:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295784:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295794:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295861:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760779.295866:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295875:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295900:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.295908:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9386 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.296051:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24220us (24790us total) trans 4294972853 rc 0/0 00000100:00100000:2.0:1789760779.296064:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5563 00000100:00100000:2.0:1789760779.296076:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760550005593 time 760587117232 state 0x2 ref 0 00000100:00100000:0.0:1789760779.299915:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.300002:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344256 00000100:00100000:2.0:1789760779.300012:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344256 00000100:00100000:2.0:1789760779.300066:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760550005593 time 760587117232 state 0x0 ref 1 00000100:00100000:2.0:1789760779.300082:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760587377648 time 760587117232 state 0x1 ref 1 00000100:00100000:2.0:1789760779.300143:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760587377648 time 760615446036 state 0x0 ref 1 00000100:00100000:2.0:1789760779.300168:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507344256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.300218:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22667264, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.300227:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.300243:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760779.300248:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.300255:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.300269:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.326469:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00002000:00010000:2.0:1789760779.326537:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22667264 -> 22671360 00010000:00010000:2.0:1789760779.326560:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326573:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326584:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326592:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.326597:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326607:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326627:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.326636:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b938d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.326767:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507344256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26596us (26858us total) trans 4294972854 rc 0/0 00000100:00100000:2.0:1789760779.326777:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5564 00000100:00100000:2.0:1789760779.326786:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760587377648 time 760615446036 state 0x2 ref 0 00000100:00100000:3.0:1789760779.333664:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.334067:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344384 00000100:00100000:1.0:1789760779.334094:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344384 00000100:00100000:1.0:1789760779.334157:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760587377648 time 760615446036 state 0x0 ref 1 00000100:00100000:1.0:1789760779.334184:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760615706452 time 760615446036 state 0x1 ref 1 00000100:00100000:1.0:1789760779.334229:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760615706452 time 760649515102 state 0x0 ref 1 00000100:00100000:1.0:1789760779.334243:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507344384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.334302:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22671360, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.334322:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.334345:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.334350:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.334356:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.334380:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.367345:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00002000:00010000:1.0:1789760779.367899:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22671360 -> 22675456 00010000:00010000:1.0:1789760779.367916:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367928:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367937:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367944:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.367949:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367957:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367978:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.367986:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9394 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.368076:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507344384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33831us (35379us total) trans 4294972855 rc 0/0 00000100:00100000:1.0:1789760779.368087:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5565 00000100:00100000:1.0:1789760779.368097:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760615706452 time 760649515102 state 0x2 ref 0 00000100:00100000:2.0:1789760779.375761:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.376077:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344512 00000100:00100000:0.0:1789760779.376087:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344512 00000100:00100000:0.0:1789760779.376136:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760615706452 time 760649515102 state 0x0 ref 1 00000100:00100000:0.0:1789760779.376150:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760649775518 time 760649515102 state 0x1 ref 1 00000100:00100000:0.0:1789760779.376181:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760649775518 time 760691484178 state 0x0 ref 1 00000100:00100000:0.0:1789760779.376193:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507344512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.376241:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22675456, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.376250:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.376266:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760779.376270:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.376276:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.376291:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.403885:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:0.0:1789760779.403971:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:0.0:1789760779.403988:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404001:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404010:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404017:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.404022:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404030:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404048:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.404055:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b939b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.404185:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507344512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27987us (28428us total) trans 4294972856 rc 0/0 00000100:00100000:0.0:1789760779.404199:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5566 00000100:00100000:0.0:1789760779.404208:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760649775518 time 760691484178 state 0x2 ref 0 00000100:00100000:3.0:1789760779.410832:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.410950:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344640 00000100:00100000:2.0:1789760779.410959:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344640 00000100:00100000:2.0:1789760779.411019:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760649775518 time 760691484178 state 0x0 ref 1 00000100:00100000:2.0:1789760779.411035:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760691744594 time 760691484178 state 0x1 ref 1 00000100:00100000:2.0:1789760779.411103:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760691744594 time 760726405881 state 0x0 ref 1 00000100:00100000:2.0:1789760779.411116:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.411164:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22679552, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.411173:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.411187:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760779.411191:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.411197:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.411213:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.437729:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:0.0:1789760779.437851:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:0.0:1789760779.437866:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437879:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437889:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437896:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.437902:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437912:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437933:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.437940:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b93a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.438032:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26913us (27210us total) trans 4294972857 rc 0/0 00000100:00100000:0.0:1789760779.438042:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5567 00000100:00100000:0.0:1789760779.438051:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760691744594 time 760726405881 state 0x2 ref 0 00000100:00100000:3.0:1789760779.441444:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.441607:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344768 00000100:00100000:1.0:1789760779.441616:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344768 00000100:00100000:1.0:1789760779.441673:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760691744594 time 760726405881 state 0x0 ref 1 00000100:00100000:1.0:1789760779.441689:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760726666297 time 760726405881 state 0x1 ref 1 00000100:00100000:1.0:1789760779.441844:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760726666297 time 760757146726 state 0x0 ref 1 00000100:00100000:1.0:1789760779.441856:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507344768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.441905:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22683648, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.441914:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.441929:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.441934:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.441940:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.441962:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.472925:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00002000:00010000:3.0:1789760779.473073:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:3.0:1789760779.473089:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473102:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473112:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473119:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760779.473124:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473133:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473152:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.473160:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b93a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.473250:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507344768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31391us (31811us total) trans 4294972858 rc 0/0 00000100:00100000:3.0:1789760779.473260:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5568 00000100:00100000:3.0:1789760779.473269:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760726666297 time 760757146726 state 0x2 ref 0 00000100:00100000:2.0:1789760779.477905:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.478012:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507344896 00000100:00100000:0.0:1789760779.478022:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507344896 00000100:00100000:0.0:1789760779.478077:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760726666297 time 760757146726 state 0x0 ref 1 00000100:00100000:0.0:1789760779.478094:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760757407142 time 760757146726 state 0x1 ref 1 00000100:00100000:0.0:1789760779.478120:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760757407142 time 760793423456 state 0x0 ref 1 00000100:00100000:0.0:1789760779.478137:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.478186:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22687744, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.478194:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.478211:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760779.478216:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.478222:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.478237:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.484317:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b702a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538290176:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484398:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484460:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b703100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538290688:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484487:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484519:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234bf5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538291200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484543:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484561:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538291712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484582:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484606:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789760779.484612:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b701500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538290304:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484635:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484654:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b702d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538290816:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484673:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484689:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234bf7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538291328:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484736:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.484758:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538291840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760779.484781:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760779.484849:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290176 00000100:00100000:0.0:1789760779.484858:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290176 00000100:00100000:2.0:1789760779.484887:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b701880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538290432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760779.484906:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:2.0:1789760779.484919:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760779.484920:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538290176:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484943:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538290944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.484978:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760779.484990:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3463:x1876699538290176:12345-0@lo:400:kworker.0 Request processed in 72us (590us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760779.484998:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:2.0:1789760779.485023:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234bf4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538291456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485050:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.485285:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538291968:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760779.485314:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.485340:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1789760779.485347:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b700000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538290560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485370:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.485388:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234bf4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538291072:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485415:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.485437:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234bf5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538291584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485464:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.485513:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290688 00000100:00100000:2.0:1789760779.485519:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290688 00000100:00100000:2.0:1789760779.485553:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290432 00000100:00100000:2.0:1789760779.485556:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290432 00000100:00100000:2.0:1789760779.485573:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:2.0:1789760779.485580:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538290688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485621:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538290688:12345-0@lo:400:kworker.0 Request processed in 40us (1132us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.485628:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789760779.485639:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291840 00000100:00100000:1.0:1789760779.485645:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291840 00000100:00100000:2.0:1789760779.485646:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:2.0:1789760779.485656:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538290432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760779.485687:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291968 00000100:00100000:1.0:1789760779.485712:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291968 00000100:00100000:2.0:1789760779.485719:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538290432:12345-0@lo:400:kworker.0 Request processed in 63us (801us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.485727:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760779.485739:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789760779.485748:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538291840:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760779.485751:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291200 00000100:00100000:2.0:1789760779.485757:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291200 00000100:00100000:2.0:1789760779.485790:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291712 00000100:00100000:2.0:1789760779.485793:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291712 00000100:00100000:2.0:1789760779.485816:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291072 00000100:00100000:2.0:1789760779.485819:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291072 00000100:00100000:2.0:1789760779.485835:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291584 00000100:00100000:2.0:1789760779.485838:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291584 00000100:00100000:2.0:1789760779.485852:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1789760779.485859:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538291200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485894:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538291200:12345-0@lo:400:kworker.0 Request processed in 35us (1350us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.485901:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1789760779.485915:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:2.0:1789760779.485924:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3463:x1876699538291712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.485954:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3463:x1876699538291712:12345-0@lo:400:kworker.0 Request processed in 32us (1371us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.485961:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:2.0:1789760779.485975:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:2.0:1789760779.485982:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538291072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.486013:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538291072:12345-0@lo:400:kworker.0 Request processed in 31us (597us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.486020:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:2.0:1789760779.486030:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1789760779.486036:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538291584:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789760779.486343:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760779.486348:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760779.486362:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1789760779.486370:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538291968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760779.486392:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760779.486432:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538291968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (1117us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760779.486439:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00002000:00000024:1.0:1789760779.486465:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760779.486491:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538291840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 745us (1709us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760779.486497:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1789760779.486845:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105822a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538291584:12345-0@lo:400:kworker.0 Request processed in 110us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.486856:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1789760779.486883:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290304 00000100:00100000:2.0:1789760779.486888:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290304 00000100:00100000:2.0:1789760779.486915:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290816 00000100:00100000:2.0:1789760779.486918:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290816 00000100:00100000:2.0:1789760779.486947:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291328 00000100:00100000:2.0:1789760779.486950:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291328 00000100:00100000:2.0:1789760779.486962:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290944 00000100:00100000:2.0:1789760779.486964:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290944 00000100:00100000:2.0:1789760779.486976:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538291456 00000100:00100000:2.0:1789760779.486978:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538291456 00000100:00100000:2.0:1789760779.486991:0:6440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538290560 00000100:00100000:2.0:1789760779.486994:0:6440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538290560 00000100:00100000:2.0:1789760779.487007:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 962 00000100:00100000:2.0:1789760779.487013:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538290304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487051:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538290304:12345-0@lo:400:kworker.0 Request processed in 38us (2414us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487115:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 962 00000100:00100000:2.0:1789760779.487128:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1789760779.487135:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538290816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487168:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538290816:12345-0@lo:400:kworker.0 Request processed in 34us (2494us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487175:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1789760779.487188:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 964 00000100:00100000:2.0:1789760779.487195:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538291328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487227:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538291328:12345-0@lo:400:kworker.0 Request processed in 34us (2491us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487234:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 964 00000100:00100000:2.0:1789760779.487247:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1789760779.487254:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538290944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487286:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbf8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538290944:12345-0@lo:400:kworker.0 Request processed in 32us (2308us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487292:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1789760779.487308:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 966 00000100:00100000:2.0:1789760779.487316:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538291456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487358:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538291456:12345-0@lo:400:kworker.0 Request processed in 43us (2307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487365:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 966 00000100:00100000:2.0:1789760779.487377:0:6440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 967 00000100:00100000:2.0:1789760779.487383:0:6440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538290560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487412:0:6440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21fbfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538290560:12345-0@lo:400:kworker.0 Request processed in 29us (2041us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.487418:0:6440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 967 00000100:00100000:2.0:1789760779.487523:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b700000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538290560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487536:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234bf4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538291072:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487546:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234bf5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538291584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487558:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b701880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538290432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487567:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b700700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538290944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487575:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234bf4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538291456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487590:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108015500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538291968:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760779.487639:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b701500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538290304:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487649:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b702d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538290816:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487659:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234bf7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538291328:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487671:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538291840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760779.487680:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b702a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a12369fc-e31f-4eec-83dc-16c04178fd0b:3463:1876699538290176:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487715:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b703100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538290688:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487734:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234bf5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538291200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760779.487747:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538291712:0@lo:400:kworker.0 00000100:00080000:1.0:1789760779.488749:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760779.488757:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760779.508977:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:1.0:1789760779.509071:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:1.0:1789760779.509086:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509098:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509107:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509114:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.509119:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509127:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509146:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.509153:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b93b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.509245:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507344896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31106us (31345us total) trans 4294972859 rc 0/0 00000100:00100000:1.0:1789760779.509255:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5569 00000100:00100000:1.0:1789760779.509265:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760757407142 time 760793423456 state 0x2 ref 0 00000100:00100000:3.0:1789760779.512862:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.513089:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345024 00000100:00100000:0.0:1789760779.513099:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345024 00000100:00100000:0.0:1789760779.513151:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760757407142 time 760793423456 state 0x0 ref 1 00000100:00100000:0.0:1789760779.513166:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760793683872 time 760793423456 state 0x1 ref 1 00000100:00100000:0.0:1789760779.513198:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760793683872 time 760828501675 state 0x0 ref 1 00000100:00100000:0.0:1789760779.513210:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.513254:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22691840, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.513263:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.513277:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760779.513281:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.513286:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.513305:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.538121:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:0.0:1789760779.538139:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538151:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538160:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538167:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.538172:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538180:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538198:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.538205:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.538307:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507345024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25093us (25449us total) trans 4294972860 rc 0/0 00000100:00100000:0.0:1789760779.538319:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5570 00000100:00100000:0.0:1789760779.538329:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760793683872 time 760828501675 state 0x2 ref 0 00000001:00080000:3.0:1789760779.538410:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00000100:00100000:3.0:1789760779.542052:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.542153:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345152 00000100:00100000:1.0:1789760779.542164:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345152 00000100:00100000:1.0:1789760779.542219:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760793683872 time 760828501675 state 0x0 ref 1 00000100:00100000:1.0:1789760779.542234:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760828762091 time 760828501675 state 0x1 ref 1 00000100:00100000:1.0:1789760779.542262:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760828762091 time 760857565675 state 0x0 ref 1 00000100:00100000:1.0:1789760779.542275:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.542323:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22695936, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.542332:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.542349:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.542354:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.542360:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.542374:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.565493:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00002000:00010000:0.0:1789760779.565800:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:0.0:1789760779.565816:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565829:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565837:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565844:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.565849:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565857:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565876:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.565888:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b93be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.565973:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23696us (23926us total) trans 4294972861 rc 0/0 00000100:00100000:0.0:1789760779.565984:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5571 00000100:00100000:0.0:1789760779.565993:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760828762091 time 760857565675 state 0x2 ref 0 00000100:00100000:1.0:1789760779.569033:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.569111:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345280 00000100:00100000:0.0:1789760779.569122:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345280 00000100:00100000:0.0:1789760779.569177:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760828762091 time 760857565675 state 0x0 ref 1 00000100:00100000:0.0:1789760779.569193:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760857826091 time 760857565675 state 0x1 ref 1 00000100:00100000:0.0:1789760779.569221:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760857826091 time 760884524441 state 0x0 ref 1 00000100:00100000:0.0:1789760779.569232:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.569281:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22700032, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.569289:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.569302:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760779.569305:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.569309:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.569324:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.599971:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00002000:00010000:1.0:1789760779.600079:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:1.0:1789760779.600095:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600109:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600118:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600126:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.600131:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600141:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600159:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600166:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.600255:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31019us (31225us total) trans 4294972862 rc 0/0 00000100:00100000:1.0:1789760779.600265:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5572 00000100:00100000:1.0:1789760779.600275:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760857826091 time 760884524441 state 0x2 ref 0 00000100:00100000:3.0:1789760779.603686:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.604244:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345408 00000100:00100000:0.0:1789760779.604254:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345408 00000100:00100000:0.0:1789760779.604303:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760857826091 time 760884524441 state 0x0 ref 1 00000100:00100000:0.0:1789760779.604316:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760884784857 time 760884524441 state 0x1 ref 1 00000100:00100000:0.0:1789760779.604341:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760884784857 time 760919644938 state 0x0 ref 1 00000100:00100000:0.0:1789760779.604351:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.604393:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22704128, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.604402:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.604416:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760779.604420:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.604426:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.604439:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.629408:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:2.0:1789760779.629428:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629441:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629451:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629458:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.629463:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629476:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629497:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.629515:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.629621:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507345408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25265us (25938us total) trans 4294972863 rc 0/0 00000100:00100000:2.0:1789760779.629632:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5573 00000100:00100000:2.0:1789760779.629643:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760884784857 time 760919644938 state 0x2 ref 0 00000001:00080000:1.0:1789760779.629730:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00000100:00100000:3.0:1789760779.632854:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.633861:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345536 00000100:00100000:1.0:1789760779.633873:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345536 00000100:00100000:1.0:1789760779.633936:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760884784857 time 760919644938 state 0x0 ref 1 00000100:00100000:1.0:1789760779.633953:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760919905354 time 760919644938 state 0x1 ref 1 00000100:00100000:1.0:1789760779.633979:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760919905354 time 760949282284 state 0x0 ref 1 00000100:00100000:1.0:1789760779.633991:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.634040:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22708224, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.634049:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.634064:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.634069:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.634076:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.634097:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.666389:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00002000:00010000:1.0:1789760779.666467:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22708224 -> 22712320 00010000:00010000:1.0:1789760779.666481:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666494:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666514:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666522:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.666527:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666537:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666658:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.666667:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.666875:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32881us (34026us total) trans 4294972864 rc 0/0 00000100:00100000:1.0:1789760779.666885:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5574 00000100:00100000:1.0:1789760779.666903:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760919905354 time 760949282284 state 0x2 ref 0 00000100:00100000:3.0:1789760779.670382:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.670540:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345664 00000100:00100000:1.0:1789760779.670550:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345664 00000100:00100000:1.0:1789760779.670605:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760919905354 time 760949282284 state 0x0 ref 1 00000100:00100000:1.0:1789760779.670621:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760949542700 time 760949282284 state 0x1 ref 1 00000100:00100000:1.0:1789760779.670656:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760949542700 time 760985959172 state 0x0 ref 1 00000100:00100000:1.0:1789760779.670667:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.670752:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22712320, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.670762:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.670789:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760779.670794:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.670801:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.670816:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.699119:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00002000:00010000:0.0:1789760779.699204:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:0.0:1789760779.699219:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699232:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699242:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699249:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.699254:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699263:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699285:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.699294:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b93da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.699415:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28745us (29039us total) trans 4294972865 rc 0/0 00000100:00100000:0.0:1789760779.699426:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5575 00000100:00100000:0.0:1789760779.699435:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760949542700 time 760985959172 state 0x2 ref 0 00000100:00100000:0.0:1789760779.701595:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.702024:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345792 00000100:00100000:2.0:1789760779.702035:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345792 00000100:00100000:2.0:1789760779.702094:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760949542700 time 760985959172 state 0x0 ref 1 00000100:00100000:2.0:1789760779.702110:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760986219588 time 760985959172 state 0x1 ref 1 00000100:00100000:2.0:1789760779.702145:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760986219588 time 761017447597 state 0x0 ref 1 00000100:00100000:2.0:1789760779.702158:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.702213:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22716416, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.702223:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.702240:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760779.702245:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.702251:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.702267:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.724606:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:0.0:1789760779.724711:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:0.0:1789760779.724728:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724742:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724752:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724760:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.724765:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724774:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724793:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.724802:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b93e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.724929:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507345792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22768us (23339us total) trans 4294972866 rc 0/0 00000100:00100000:0.0:1789760779.724942:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5576 00000100:00100000:0.0:1789760779.724952:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760986219588 time 761017447597 state 0x2 ref 0 00000100:00100000:3.0:1789760779.728652:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.733646:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507345920 00000100:00100000:1.0:1789760779.733664:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507345920 00000100:00100000:1.0:1789760779.733807:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 760986219588 time 761017447597 state 0x0 ref 1 00000100:00100000:1.0:1789760779.733822:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761017708013 time 761017447597 state 0x1 ref 1 00000100:00100000:1.0:1789760779.733848:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761017708013 time 761049151935 state 0x0 ref 1 00000100:00100000:1.0:1789760779.733860:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.733913:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22720512, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.733922:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.733938:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760779.733943:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.733949:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.733963:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.758770:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00002000:00010000:0.0:1789760779.758956:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:0.0:1789760779.758972:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.758988:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.758998:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.759005:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760779.759010:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.759020:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.759042:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.759051:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b93e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.759149:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507345920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25286us (30501us total) trans 4294972867 rc 0/0 00000100:00100000:0.0:1789760779.759161:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5577 00000100:00100000:0.0:1789760779.759171:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761017708013 time 761049151935 state 0x2 ref 0 00000100:00100000:3.0:1789760779.764631:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.764734:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346048 00000100:00100000:0.0:1789760779.764747:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346048 00000100:00100000:0.0:1789760779.764801:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761017708013 time 761049151935 state 0x0 ref 1 00000100:00100000:0.0:1789760779.764813:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761049412351 time 761049151935 state 0x1 ref 1 00000100:00100000:0.0:1789760779.764838:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761049412351 time 761080142625 state 0x0 ref 1 00000100:00100000:0.0:1789760779.764846:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.764883:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22724608, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.764890:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.764902:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760779.764907:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.764913:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.764927:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.809912:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00002000:00010000:1.0:1789760779.810201:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:1.0:1789760779.810217:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810230:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810239:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810245:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.810251:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810261:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810280:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.810289:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b93ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.810428:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45570us (45801us total) trans 4294972868 rc 0/0 00000100:00100000:1.0:1789760779.810437:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5578 00000100:00100000:1.0:1789760779.810444:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761049412351 time 761080142625 state 0x2 ref 0 00000100:00100000:3.0:1789760779.815244:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.816054:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346176 00000100:00100000:0.0:1789760779.816065:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346176 00000100:00100000:0.0:1789760779.816124:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761049412351 time 761080142625 state 0x0 ref 1 00000100:00100000:0.0:1789760779.816139:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761080403041 time 761080142625 state 0x1 ref 1 00000100:00100000:0.0:1789760779.816213:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761080403041 time 761131515529 state 0x0 ref 1 00000100:00100000:0.0:1789760779.816227:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.816276:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22728704, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.816285:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.816301:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760779.816306:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.816313:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.816328:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.854020:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00002000:00010000:1.0:1789760779.854639:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:1.0:1789760779.854659:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854675:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854686:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854729:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760779.854801:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854813:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854840:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.854849:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b93f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.854967:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38735us (39724us total) trans 4294972869 rc 0/0 00000100:00100000:1.0:1789760779.854980:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5579 00000100:00100000:1.0:1789760779.854993:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761080403041 time 761131515529 state 0x2 ref 0 00000100:00100000:2.0:1789760779.857959:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760779.858095:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346304 00000100:00100000:0.0:1789760779.858108:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346304 00000100:00100000:0.0:1789760779.858185:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761080403041 time 761131515529 state 0x0 ref 1 00000100:00100000:0.0:1789760779.858205:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761131775945 time 761131515529 state 0x1 ref 1 00000100:00100000:0.0:1789760779.858239:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761131775945 time 761173541331 state 0x0 ref 1 00000100:00100000:0.0:1789760779.858254:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760779.858320:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22732800, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:0.0:1789760779.858332:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.858352:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760779.858359:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.858367:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.858386:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.896454:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00002000:00010000:2.0:1789760779.896713:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:2.0:1789760779.896727:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896740:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896749:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896757:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.896762:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896776:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896795:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.896803:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b93fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.896894:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38638us (38947us total) trans 4294972870 rc 0/0 00000100:00100000:2.0:1789760779.896903:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5580 00000100:00100000:2.0:1789760779.896913:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761131775945 time 761173541331 state 0x2 ref 0 00000100:00100000:3.0:1789760779.901245:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.901385:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346432 00000100:00100000:1.0:1789760779.901395:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346432 00000100:00100000:1.0:1789760779.901447:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761131775945 time 761173541331 state 0x0 ref 1 00000100:00100000:1.0:1789760779.901462:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761173801747 time 761173541331 state 0x1 ref 1 00000100:00100000:1.0:1789760779.901536:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761173801747 time 761216838557 state 0x0 ref 1 00000100:00100000:1.0:1789760779.901556:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.901606:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22736896, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.901616:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.901632:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760779.901637:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.901643:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.901654:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.938072:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00002000:00010000:2.0:1789760779.938250:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:2.0:1789760779.938266:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938280:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938290:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938297:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.938303:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938312:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938331:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.938339:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9404 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.938442:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36882us (37198us total) trans 4294972871 rc 0/0 00000100:00100000:2.0:1789760779.938452:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5581 00000100:00100000:2.0:1789760779.938462:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761173801747 time 761216838557 state 0x2 ref 0 00000100:00100000:0.0:1789760779.944512:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760779.944631:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346560 00000100:00100000:1.0:1789760779.944646:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346560 00000100:00100000:1.0:1789760779.944736:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761173801747 time 761216838557 state 0x0 ref 1 00000100:00100000:1.0:1789760779.944752:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761217098973 time 761216838557 state 0x1 ref 1 00000100:00100000:1.0:1789760779.944784:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761217098973 time 761260087558 state 0x0 ref 1 00000100:00100000:1.0:1789760779.944798:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760779.944942:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22740992, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:1.0:1789760779.944952:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.944968:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760779.944973:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.944979:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.944993:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.991316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00002000:00010000:2.0:1789760779.991425:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22740992 -> 22745088 00010000:00010000:2.0:1789760779.991447:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991463:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991473:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991486:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760779.991492:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991512:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991656:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.991667:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b940b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.992049:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47245us (47555us total) trans 4294972872 rc 0/0 00000100:00100000:2.0:1789760779.992063:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5582 00000100:00100000:2.0:1789760779.992076:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761217098973 time 761260087558 state 0x2 ref 0 00000100:00100000:3.0:1789760779.998360:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760779.998448:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346688 00000100:00100000:2.0:1789760779.998468:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346688 00000100:00100000:2.0:1789760779.998568:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761217098973 time 761260087558 state 0x0 ref 1 00000100:00100000:2.0:1789760779.998660:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761260347974 time 761260087558 state 0x1 ref 1 00000100:00100000:2.0:1789760779.998812:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761260347974 time 761314112565 state 0x0 ref 1 00000100:00100000:2.0:1789760779.998832:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760779.998903:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22745088, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xb007 00010000:00010000:2.0:1789760779.998915:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.998936:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760779.998942:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.998950:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.998969:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.057654:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22745088 -> 22749184 00002000:00010000:2.0:1789760780.057666:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:2.0:1789760780.057669:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:2.0:1789760780.057686:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057743:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057755:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057763:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.057771:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057781:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057808:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.057815:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9412 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.057932:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507346688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59096us (59574us total) trans 4294972873 rc 0/0 00000100:00100000:2.0:1789760780.057945:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5583 00000100:00100000:2.0:1789760780.057957:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761260347974 time 761314112565 state 0x2 ref 0 00000001:00080000:1.0:1789760780.058358:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00000100:00100000:1.0:1789760780.061334:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.061553:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346816 00000100:00100000:2.0:1789760780.061566:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346816 00000100:00100000:2.0:1789760780.061642:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761260347974 time 761314112565 state 0x0 ref 1 00000100:00100000:2.0:1789760780.061660:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761314372981 time 761314112565 state 0x1 ref 1 00000100:00100000:2.0:1789760780.061872:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761314372981 time 761377173052 state 0x0 ref 1 00000100:00100000:2.0:1789760780.061893:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507346816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.061959:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22749184, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.061969:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.061988:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.061994:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.062002:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.062109:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.095776:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22749184 -> 22753280 00010000:00010000:1.0:1789760780.095802:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095816:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095836:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095846:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760780.095852:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095863:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095889:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.095897:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9419 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.096015:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507346816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34123us (34690us total) trans 4294972874 rc 0/0 00000100:00100000:1.0:1789760780.096026:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5584 00000100:00100000:1.0:1789760780.096037:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761314372981 time 761377173052 state 0x2 ref 0 00000001:00080000:3.0:1789760780.096171:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00000100:00100000:0.0:1789760780.103528:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.104158:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507346944 00000100:00100000:2.0:1789760780.104171:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507346944 00000100:00100000:2.0:1789760780.104263:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761314372981 time 761377173052 state 0x0 ref 1 00000100:00100000:2.0:1789760780.104294:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761377433468 time 761377173052 state 0x1 ref 1 00000100:00100000:2.0:1789760780.104329:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761377433468 time 761419632084 state 0x0 ref 1 00000100:00100000:2.0:1789760780.104343:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.104410:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22753280, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.104422:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.104441:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.104446:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.104455:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.104474:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.168642:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00002000:00010000:3.0:1789760780.169359:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22753280 -> 22757376 00010000:00010000:3.0:1789760780.169380:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169397:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169422:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169430:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.169437:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169447:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169473:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.169481:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9420 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.170256:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507346944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65905us (66963us total) trans 4294972875 rc 0/0 00000100:00100000:3.0:1789760780.170270:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5585 00000100:00100000:3.0:1789760780.170284:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761377433468 time 761419632084 state 0x2 ref 0 00000100:00100000:3.0:1789760780.174982:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.175326:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347072 00000100:00100000:0.0:1789760780.175341:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347072 00000100:00100000:0.0:1789760780.175432:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761377433468 time 761419632084 state 0x0 ref 1 00000100:00100000:0.0:1789760780.175452:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761419892500 time 761419632084 state 0x1 ref 1 00000100:00100000:0.0:1789760780.175491:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761419892500 time 761490793389 state 0x0 ref 1 00000100:00100000:0.0:1789760780.175514:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.175648:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22757376, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.175659:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.175678:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760780.175683:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.175831:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.175850:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.188241:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760780.224438:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00002000:00010000:2.0:1789760780.224669:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:2.0:1789760780.224689:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224723:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224734:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224743:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.224749:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224759:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224783:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.224793:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9427 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.224962:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49443us (49989us total) trans 4294972876 rc 0/0 00000100:00100000:2.0:1789760780.224976:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5586 00000100:00100000:2.0:1789760780.224989:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761419892500 time 761490793389 state 0x2 ref 0 00000100:00100000:1.0:1789760780.229314:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.229481:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347200 00000100:00100000:2.0:1789760780.229493:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347200 00000100:00100000:2.0:1789760780.229637:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761419892500 time 761490793389 state 0x0 ref 1 00000100:00100000:2.0:1789760780.229657:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761491053805 time 761490793389 state 0x1 ref 1 00000100:00100000:2.0:1789760780.229771:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761491053805 time 761545072067 state 0x0 ref 1 00000100:00100000:2.0:1789760780.229786:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.229856:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22761472, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.229868:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.229888:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.229893:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.229900:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.229917:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.263169:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:2.0:1789760780.263318:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:2.0:1789760780.263341:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263357:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263367:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263375:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.263381:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263390:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263414:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.263426:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b942e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.263557:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33766us (34250us total) trans 4294972877 rc 0/0 00000100:00100000:2.0:1789760780.263573:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5587 00000100:00100000:2.0:1789760780.263585:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761491053805 time 761545072067 state 0x2 ref 0 00000100:00100000:0.0:1789760780.268127:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.268222:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347328 00000100:00100000:3.0:1789760780.268237:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347328 00000100:00100000:3.0:1789760780.268295:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761491053805 time 761545072067 state 0x0 ref 1 00000100:00100000:3.0:1789760780.268312:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761545332483 time 761545072067 state 0x1 ref 1 00000100:00100000:3.0:1789760780.268345:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761545332483 time 761583648593 state 0x0 ref 1 00000100:00100000:3.0:1789760780.268360:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.268482:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22765568, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.268492:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.268524:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.268528:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.268541:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.269778:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.298882:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:0.0:1789760780.298901:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298914:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298924:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298937:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.298942:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298951:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298972:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298980:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9435 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.299150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00000100:00100000:0.0:1789760780.299206:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30843us (31091us total) trans 4294972878 rc 0/0 00000100:00100000:0.0:1789760780.299217:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5588 00000100:00100000:0.0:1789760780.299227:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761545332483 time 761583648593 state 0x2 ref 0 00000100:00100000:0.0:1789760780.303222:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.303291:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347456 00000100:00100000:2.0:1789760780.303299:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347456 00000100:00100000:2.0:1789760780.303346:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761545332483 time 761583648593 state 0x0 ref 1 00000100:00100000:2.0:1789760780.303361:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761583909009 time 761583648593 state 0x1 ref 1 00000100:00100000:2.0:1789760780.303388:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761583909009 time 761618691858 state 0x0 ref 1 00000100:00100000:2.0:1789760780.303398:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.303449:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22769664, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.303458:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.303473:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.303478:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.303484:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.303505:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.366859:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00002000:00010000:3.0:1789760780.367275:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:3.0:1789760780.367291:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367304:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367314:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367321:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.367326:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367336:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367354:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.367361:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b943c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.367627:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64224us (64409us total) trans 4294972879 rc 0/0 00000100:00100000:3.0:1789760780.367638:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5589 00000100:00100000:3.0:1789760780.367647:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761583909009 time 761618691858 state 0x2 ref 0 00000100:00100000:1.0:1789760780.375203:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.375343:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347584 00000100:00100000:3.0:1789760780.375352:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347584 00000100:00100000:3.0:1789760780.375394:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761583909009 time 761618691858 state 0x0 ref 1 00000100:00100000:3.0:1789760780.375406:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761618952274 time 761618691858 state 0x1 ref 1 00000100:00100000:3.0:1789760780.375434:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761618952274 time 761690736992 state 0x0 ref 1 00000100:00100000:3.0:1789760780.375621:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507347584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.375670:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22773760, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.375817:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.375833:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.375837:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.375843:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.375857:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.425936:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:3.0:1789760780.426123:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:3.0:1789760780.426139:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426152:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426161:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426168:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.426173:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426182:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426212:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.426220:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9443 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.426306:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507347584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50805us (51113us total) trans 4294972880 rc 0/0 00000100:00100000:3.0:1789760780.426316:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5590 00000100:00100000:3.0:1789760780.426325:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761618952274 time 761690736992 state 0x2 ref 0 00000100:00100000:0.0:1789760780.432123:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.432253:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347712 00000100:00100000:3.0:1789760780.432263:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347712 00000100:00100000:3.0:1789760780.432315:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761618952274 time 761690736992 state 0x0 ref 1 00000100:00100000:3.0:1789760780.432329:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761690997408 time 761690736992 state 0x1 ref 1 00000100:00100000:3.0:1789760780.432360:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761690997408 time 761747663176 state 0x0 ref 1 00000100:00100000:3.0:1789760780.432372:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.432416:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22777856, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.432425:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.432440:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.432445:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.432450:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.432469:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.467871:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00002000:00010000:3.0:1789760780.468045:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:3.0:1789760780.468058:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468068:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468074:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468078:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.468082:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468088:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468103:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.468107:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b944a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.468399:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36015us (36273us total) trans 4294972881 rc 0/0 00000100:00100000:3.0:1789760780.468409:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5591 00000100:00100000:3.0:1789760780.468418:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761690997408 time 761747663176 state 0x2 ref 0 00000100:00100000:1.0:1789760780.478257:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.478341:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347840 00000100:00100000:3.0:1789760780.478351:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347840 00000100:00100000:3.0:1789760780.478401:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761690997408 time 761747663176 state 0x0 ref 1 00000100:00100000:3.0:1789760780.478426:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761747923592 time 761747663176 state 0x1 ref 1 00000100:00100000:3.0:1789760780.478453:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761747923592 time 761793755926 state 0x0 ref 1 00000100:00100000:3.0:1789760780.478465:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507347840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.478523:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22781952, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.478532:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478548:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.478552:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.478558:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478572:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.507946:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760780.507970:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538292096:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760780.508035:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760780.508266:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292096 00000100:00100000:0.0:1789760780.508274:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292096 00000100:00100000:0.0:1789760780.508329:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:0.0:1789760780.508337:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538292096:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760780.508360:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3835691008 avail 3791650816 00000100:00100000:0.0:1789760780.508417:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538292096:12345-0@lo:13:osp-pre-0-1.0 Request processed in 80us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760780.508424:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1789760780.508557:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538292096:0@lo:13:osp-pre-0-1.0 00000001:00080000:1.0:1789760780.509989:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:2.0:1789760780.510168:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22781952 -> 22786048 00000100:00100000:3.0:1789760780.510514:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538292224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760780.510581:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760780.510750:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292224 00000100:00100000:0.0:1789760780.510757:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292224 00000100:00100000:0.0:1789760780.510806:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789760780.510815:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538292224:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760780.510835:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3835691008 avail 3790602240 00000100:00100000:0.0:1789760780.510879:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538292224:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760780.510886:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789760780.510928:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538292224:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789760780.511012:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511024:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511032:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511040:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.511044:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511053:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511073:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.511081:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9451 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.511268:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507347840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32802us (33018us total) trans 4294972882 rc 0/0 00000100:00100000:2.0:1789760780.511279:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5592 00000100:00100000:2.0:1789760780.511289:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761747923592 time 761793755926 state 0x2 ref 0 00000100:00100000:1.0:1789760780.517057:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.517162:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507347968 00000100:00100000:3.0:1789760780.517173:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507347968 00000100:00100000:3.0:1789760780.517222:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761747923592 time 761793755926 state 0x0 ref 1 00000100:00100000:3.0:1789760780.517236:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761794016342 time 761793755926 state 0x1 ref 1 00000100:00100000:3.0:1789760780.517310:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761794016342 time 761832612568 state 0x0 ref 1 00000100:00100000:3.0:1789760780.517321:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.517368:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22786048, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.517376:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.517401:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.517405:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.517412:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.517424:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.566874:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:3.0:1789760780.566892:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566905:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566914:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566922:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.566927:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566936:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566953:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.566960:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9458 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.567094:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00000100:00100000:3.0:1789760780.567146:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507347968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49821us (50092us total) trans 4294972883 rc 0/0 00000100:00100000:3.0:1789760780.567156:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5593 00000100:00100000:3.0:1789760780.567165:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761794016342 time 761832612568 state 0x2 ref 0 00000100:00100000:0.0:1789760780.573800:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760780.573935:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348096 00000100:00100000:3.0:1789760780.573944:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348096 00000100:00100000:3.0:1789760780.574000:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761794016342 time 761832612568 state 0x0 ref 1 00000100:00100000:3.0:1789760780.574017:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761832872984 time 761832612568 state 0x1 ref 1 00000100:00100000:3.0:1789760780.574047:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761832872984 time 761889350380 state 0x0 ref 1 00000100:00100000:3.0:1789760780.574060:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.574106:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22790144, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.574115:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.574130:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760780.574134:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.574141:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.574154:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.599970:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:0.0:1789760780.599987:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600001:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600009:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600017:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.600023:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600032:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600050:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.600058:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b945f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.600144:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507348096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26082us (26348us total) trans 4294972884 rc 0/0 00000100:00100000:0.0:1789760780.600153:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5594 00000100:00100000:0.0:1789760780.600162:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761832872984 time 761889350380 state 0x2 ref 0 00000001:00080000:1.0:1789760780.601345:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00000100:00100000:3.0:1789760780.605940:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.606132:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348224 00000100:00100000:2.0:1789760780.606143:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348224 00000100:00100000:2.0:1789760780.606199:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761832872984 time 761889350380 state 0x0 ref 1 00000100:00100000:2.0:1789760780.606214:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761889610796 time 761889350380 state 0x1 ref 1 00000100:00100000:2.0:1789760780.606238:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761889610796 time 761921542245 state 0x0 ref 1 00000100:00100000:2.0:1789760780.606250:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.606299:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22794240, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.606315:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.606330:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.606335:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.606341:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.606357:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.638505:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:0.0:1789760780.638525:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.638538:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.638548:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.638560:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.638565:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.638575:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.638587:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00010000:00010000:0.0:1789760780.638594:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.638601:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9466 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.638778:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32439us (32757us total) trans 4294972885 rc 0/0 00000100:00100000:0.0:1789760780.638790:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5595 00000100:00100000:0.0:1789760780.638800:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761889610796 time 761921542245 state 0x2 ref 0 00000100:00100000:3.0:1789760780.641188:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.641302:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348352 00000100:00100000:2.0:1789760780.641313:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348352 00000100:00100000:2.0:1789760780.641367:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761889610796 time 761921542245 state 0x0 ref 1 00000100:00100000:2.0:1789760780.641380:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761921802661 time 761921542245 state 0x1 ref 1 00000100:00100000:2.0:1789760780.641405:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761921802661 time 761956709676 state 0x0 ref 1 00000100:00100000:2.0:1789760780.641414:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760780.641459:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22798336, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:2.0:1789760780.641470:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.641486:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760780.641491:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.641511:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.641526:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.666263:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:0.0:1789760780.666289:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666303:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666320:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666337:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.666341:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666351:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666370:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666378:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b946d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.666414:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00000100:00100000:0.0:1789760780.666518:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25099us (25333us total) trans 4294972886 rc 0/0 00000100:00100000:0.0:1789760780.666528:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5596 00000100:00100000:0.0:1789760780.666651:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761921802661 time 761956709676 state 0x2 ref 0 00000100:00100000:3.0:1789760780.670979:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.671030:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348480 00000100:00100000:0.0:1789760780.671042:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348480 00000100:00100000:0.0:1789760780.671099:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761921802661 time 761956709676 state 0x0 ref 1 00000100:00100000:0.0:1789760780.671115:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761956970092 time 761956709676 state 0x1 ref 1 00000100:00100000:0.0:1789760780.671148:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761956970092 time 761986451096 state 0x0 ref 1 00000100:00100000:0.0:1789760780.671161:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.671211:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22802432, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.671220:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.671235:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760780.671240:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.671246:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.671261:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.708856:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00002000:00010000:3.0:1789760780.710234:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22802432 -> 22806528 00010000:00010000:3.0:1789760780.710251:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710269:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710278:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710286:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760780.710290:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710299:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710322:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.710329:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9474 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.710776:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39611us (39803us total) trans 4294972887 rc 0/0 00000100:00100000:3.0:1789760780.710787:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5597 00000100:00100000:3.0:1789760780.710797:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761956970092 time 761986451096 state 0x2 ref 0 00000100:00100000:3.0:1789760780.719000:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.719093:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348608 00000100:00100000:0.0:1789760780.719103:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348608 00000100:00100000:0.0:1789760780.719156:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761956970092 time 761986451096 state 0x0 ref 1 00000100:00100000:0.0:1789760780.719172:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761986711512 time 761986451096 state 0x1 ref 1 00000100:00100000:0.0:1789760780.719200:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761986711512 time 762034504266 state 0x0 ref 1 00000100:00100000:0.0:1789760780.719211:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.719256:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22806528, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.719266:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.719280:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760780.719284:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.719290:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.719303:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.754286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:0.0:1789760780.754404:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:0.0:1789760780.754419:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754433:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754442:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754449:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.754454:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754463:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754491:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.754508:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b947b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.754671:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35455us (35674us total) trans 4294972888 rc 0/0 00000100:00100000:0.0:1789760780.754682:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5598 00000100:00100000:0.0:1789760780.755199:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761986711512 time 762034504266 state 0x2 ref 0 00000100:00100000:1.0:1789760780.757731:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.757967:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348736 00000100:00100000:0.0:1789760780.757976:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348736 00000100:00100000:0.0:1789760780.758042:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 761986711512 time 762034504266 state 0x0 ref 1 00000100:00100000:0.0:1789760780.758056:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762034764682 time 762034504266 state 0x1 ref 1 00000100:00100000:0.0:1789760780.758084:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762034764682 time 762073387377 state 0x0 ref 1 00000100:00100000:0.0:1789760780.758095:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.758140:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22810624, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.758149:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.758164:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760780.758169:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.758175:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.758188:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.787637:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00002000:00010000:0.0:1789760780.787793:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:0.0:1789760780.787809:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787822:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787831:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787838:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.787843:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787922:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.788066:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.788079:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9482 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.788206:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507348736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30107us (30519us total) trans 4294972889 rc 0/0 00000100:00100000:0.0:1789760780.788216:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5599 00000100:00100000:0.0:1789760780.788225:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762034764682 time 762073387377 state 0x2 ref 0 00000100:00100000:3.0:1789760780.793785:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.793905:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348864 00000100:00100000:0.0:1789760780.793915:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348864 00000100:00100000:0.0:1789760780.793968:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762034764682 time 762073387377 state 0x0 ref 1 00000100:00100000:0.0:1789760780.793984:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762073647793 time 762073387377 state 0x1 ref 1 00000100:00100000:0.0:1789760780.794005:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762073647793 time 762109309190 state 0x0 ref 1 00000100:00100000:0.0:1789760780.794016:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.794067:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22814720, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.794076:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.794092:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760780.794097:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.794103:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.794122:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.824434:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:2.0:1789760780.824718:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:2.0:1789760780.824733:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824745:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824753:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824760:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.824765:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824774:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824793:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.824801:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9489 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.825150:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507348864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31129us (31370us total) trans 4294972890 rc 0/0 00000100:00100000:2.0:1789760780.825163:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5600 00000100:00100000:2.0:1789760780.825172:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762073647793 time 762109309190 state 0x2 ref 0 00000100:00100000:1.0:1789760780.834988:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.835132:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507348992 00000100:00100000:0.0:1789760780.835144:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507348992 00000100:00100000:0.0:1789760780.835206:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762073647793 time 762109309190 state 0x0 ref 1 00000100:00100000:0.0:1789760780.835233:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762109569606 time 762109309190 state 0x1 ref 1 00000100:00100000:0.0:1789760780.835264:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762109569606 time 762150566659 state 0x0 ref 1 00000100:00100000:0.0:1789760780.835275:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507348992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.835324:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22818816, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.835333:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.835350:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760780.835354:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.835361:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.835375:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.889388:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00002000:00010000:0.0:1789760780.889611:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:0.0:1789760780.889629:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889641:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889649:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889656:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.889662:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889671:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889690:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.889724:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9490 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.890010:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507348992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54730us (55030us total) trans 4294972891 rc 0/0 00000100:00100000:0.0:1789760780.890020:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5601 00000100:00100000:0.0:1789760780.890030:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762109569606 time 762150566659 state 0x2 ref 0 00000100:00100000:3.0:1789760780.899895:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.900009:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349120 00000100:00100000:0.0:1789760780.900019:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349120 00000100:00100000:0.0:1789760780.900072:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762109569606 time 762150566659 state 0x0 ref 1 00000100:00100000:0.0:1789760780.900087:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762150827075 time 762150566659 state 0x1 ref 1 00000100:00100000:0.0:1789760780.900120:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762150827075 time 762215422878 state 0x0 ref 1 00000100:00100000:0.0:1789760780.900134:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507349120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760780.900181:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22822912, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:0.0:1789760780.900191:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.900206:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760780.900211:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.900217:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.900231:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.940467:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00002000:00010000:0.0:1789760780.940667:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22822912 -> 22827008 00010000:00010000:0.0:1789760780.940681:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940714:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940724:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940731:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760780.940735:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940743:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940761:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.940781:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9497 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.940873:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507349120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40738us (40983us total) trans 4294972892 rc 0/0 00000100:00100000:0.0:1789760780.940883:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5602 00000100:00100000:0.0:1789760780.940976:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762150827075 time 762215422878 state 0x2 ref 0 00000100:00100000:3.0:1789760780.943988:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760780.946754:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349248 00000100:00100000:2.0:1789760780.946827:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349248 00000100:00100000:3.0:1789760780.947040:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762150827075 time 762215422878 state 0x0 ref 1 00000100:00100000:3.0:1789760780.947056:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762215683294 time 762215422878 state 0x1 ref 1 00000100:00100000:3.0:1789760780.947118:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762215683294 time 762262419997 state 0x0 ref 1 00000100:00100000:3.0:1789760780.947134:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760780.947184:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22827008, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:3.0:1789760780.947193:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.947209:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760780.947214:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.947220:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.947236:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.983223:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00002000:00010000:2.0:1789760780.983288:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:2.0:1789760780.983303:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983321:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983337:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983344:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760780.983349:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983357:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983376:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983383:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b949e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.983520:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36386us (39538us total) trans 4294972893 rc 0/0 00000100:00100000:2.0:1789760780.983540:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5603 00000100:00100000:2.0:1789760780.983550:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762215683294 time 762262419997 state 0x2 ref 0 00000100:00100000:2.0:1789760780.991672:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760780.992051:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349376 00000100:00100000:0.0:1789760780.992062:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349376 00000100:00100000:0.0:1789760780.992122:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762215683294 time 762262419997 state 0x0 ref 1 00000100:00100000:0.0:1789760780.992140:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762262680413 time 762262419997 state 0x1 ref 1 00000100:00100000:1.0:1789760780.992331:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762262680413 time 762307631090 state 0x0 ref 1 00000100:00100000:1.0:1789760780.992347:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507349376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760780.992398:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22831104, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xb007 00010000:00010000:1.0:1789760780.992407:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.992422:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760780.992426:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.992432:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.992446:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.021184:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760781.021208:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538292352:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760781.021282:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760781.021482:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292352 00000100:00100000:3.0:1789760781.021500:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292352 00000100:00100000:3.0:1789760781.021687:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1789760781.021717:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538292352:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760781.021759:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760781.021819:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538292352:12345-0@lo:41:osp-pre-1-0.0 Request processed in 104us (536us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760781.021827:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1789760781.021869:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538292352:0@lo:41:osp-pre-1-0.0 00002000:00010000:1.0:1789760781.023504:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22831104 -> 22835200 00002000:00010000:1.0:1789760781.023513:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:1.0:1789760781.023515:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:1.0:1789760781.023536:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023548:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023556:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023564:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.023568:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023577:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023595:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.023601:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b94a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.023679:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507349376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31333us (32015us total) trans 4294972894 rc 0/0 00000100:00100000:1.0:1789760781.023689:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5604 00000100:00100000:1.0:1789760781.023728:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762262680413 time 762307631090 state 0x2 ref 0 00000001:00080000:2.0:1789760781.023791:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00000100:00100000:3.0:1789760781.026344:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760781.026431:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349504 00000100:00100000:0.0:1789760781.026440:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349504 00000100:00100000:0.0:1789760781.026509:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762262680413 time 762307631090 state 0x0 ref 1 00000100:00100000:0.0:1789760781.026524:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762307891506 time 762307631090 state 0x1 ref 1 00000100:00100000:0.0:1789760781.026592:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762307891506 time 762341894898 state 0x0 ref 1 00000100:00100000:0.0:1789760781.026605:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760781.026655:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22835200, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:0.0:1789760781.026664:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.026680:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760781.026685:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.026717:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.026733:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760781.026796:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760781.050089:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:3.0:1789760781.050198:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22835200 -> 22839296 00010000:00010000:3.0:1789760781.050231:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050245:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050254:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050262:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.050267:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050276:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050358:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.050366:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b94ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.050537:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23930us (24199us total) trans 4294972895 rc 0/0 00000100:00100000:3.0:1789760781.050549:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5605 00000100:00100000:3.0:1789760781.050558:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762307891506 time 762341894898 state 0x2 ref 0 00000100:00100000:3.0:1789760781.053880:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760781.053979:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349632 00000100:00100000:2.0:1789760781.053989:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349632 00000100:00100000:2.0:1789760781.054045:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762307891506 time 762341894898 state 0x0 ref 1 00000100:00100000:2.0:1789760781.054060:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762342155314 time 762341894898 state 0x1 ref 1 00000100:00100000:2.0:1789760781.054090:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762342155314 time 762369393658 state 0x0 ref 1 00000100:00100000:2.0:1789760781.054102:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507349632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.054144:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22839296, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.054152:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.054167:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.054171:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.054177:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.054189:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.083167:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00002000:00010000:3.0:1789760781.083859:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:3.0:1789760781.083875:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083887:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083895:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083903:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.083908:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083917:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083936:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.083948:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b94b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.084079:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507349632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29973us (30203us total) trans 4294972896 rc 0/0 00000100:00100000:3.0:1789760781.084089:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5606 00000100:00100000:3.0:1789760781.084098:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762342155314 time 762369393658 state 0x2 ref 0 00000100:00100000:2.0:1789760781.087320:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.087798:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349760 00000100:00100000:1.0:1789760781.087814:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349760 00000100:00100000:1.0:1789760781.087867:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762342155314 time 762369393658 state 0x0 ref 1 00000100:00100000:1.0:1789760781.087881:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762369654074 time 762369393658 state 0x1 ref 1 00000100:00100000:1.0:1789760781.087910:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762369654074 time 762403213502 state 0x0 ref 1 00000100:00100000:1.0:1789760781.087923:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760781.087968:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22843392, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760781.087976:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.087991:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760781.087995:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.088002:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.088020:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.127368:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00002000:00010000:3.0:1789760781.127475:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:3.0:1789760781.127498:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127511:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127634:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127642:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.127646:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127656:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127682:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.127689:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b94ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.128058:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507349760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40131us (40941us total) trans 4294972897 rc 0/0 00000100:00100000:3.0:1789760781.128070:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5607 00000100:00100000:3.0:1789760781.128080:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762369654074 time 762403213502 state 0x2 ref 0 00000100:00100000:1.0:1789760781.133149:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.133851:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507349888 00000100:00100000:3.0:1789760781.133863:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507349888 00000100:00100000:3.0:1789760781.133915:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762369654074 time 762403213502 state 0x0 ref 1 00000100:00100000:3.0:1789760781.133940:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762403473918 time 762403213502 state 0x1 ref 1 00000100:00100000:3.0:1789760781.133970:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762403473918 time 762449272949 state 0x0 ref 1 00000100:00100000:3.0:1789760781.133982:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507349888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.134027:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22847488, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.134036:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.134051:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.134055:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.134062:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.134076:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.165870:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00002000:00010000:3.0:1789760781.165999:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22847488 -> 22851584 00010000:00010000:3.0:1789760781.166012:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166021:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166028:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166032:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.166036:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166042:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166056:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166072:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.166165:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507349888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32180us (33021us total) trans 4294972898 rc 0/0 00000100:00100000:3.0:1789760781.166175:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5608 00000100:00100000:3.0:1789760781.166185:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762403473918 time 762449272949 state 0x2 ref 0 00000100:00100000:0.0:1789760781.174873:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.175054:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350016 00000100:00100000:3.0:1789760781.175064:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350016 00000100:00100000:3.0:1789760781.175121:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762403473918 time 762449272949 state 0x0 ref 1 00000100:00100000:3.0:1789760781.175135:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762449533365 time 762449272949 state 0x1 ref 1 00000100:00100000:3.0:1789760781.175170:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762449533365 time 762490473390 state 0x0 ref 1 00000100:00100000:3.0:1789760781.175183:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507350016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.175230:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22851584, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.175239:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.175254:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.175259:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.175266:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.175285:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.202608:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:3.0:1789760781.202873:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:3.0:1789760781.202889:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202901:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202910:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202918:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.202923:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202932:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202952:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.202960:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.203058:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507350016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27872us (28187us total) trans 4294972899 rc 0/0 00000100:00100000:3.0:1789760781.203076:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5609 00000100:00100000:3.0:1789760781.203086:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762449533365 time 762490473390 state 0x2 ref 0 00000100:00100000:2.0:1789760781.207836:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.207940:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350144 00000100:00100000:3.0:1789760781.207958:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350144 00000100:00100000:3.0:1789760781.208009:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762449533365 time 762490473390 state 0x0 ref 1 00000100:00100000:3.0:1789760781.208024:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762490733806 time 762490473390 state 0x1 ref 1 00000100:00100000:2.0:1789760781.208146:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762490733806 time 762523446619 state 0x0 ref 1 00000100:00100000:2.0:1789760781.208170:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507350144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.208221:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22855680, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.208230:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.208247:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.208251:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.208258:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.208272:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.212451:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760781.238814:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00002000:00010000:3.0:1789760781.238898:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:3.0:1789760781.238915:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238928:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238937:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238943:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.238948:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238956:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238976:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.238984:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b94cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.239113:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507350144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30941us (31314us total) trans 4294972900 rc 0/0 00000100:00100000:3.0:1789760781.239123:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5610 00000100:00100000:3.0:1789760781.239133:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762490733806 time 762523446619 state 0x2 ref 0 00000100:00100000:1.0:1789760781.245681:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.245813:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350272 00000100:00100000:3.0:1789760781.245824:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350272 00000100:00100000:2.0:1789760781.246060:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762490733806 time 762523446619 state 0x0 ref 1 00000100:00100000:2.0:1789760781.246144:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762523707035 time 762523446619 state 0x1 ref 1 00000100:00100000:2.0:1789760781.246221:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762523707035 time 762561523728 state 0x0 ref 1 00000100:00100000:2.0:1789760781.246236:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507350272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.246284:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22859776, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.246293:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.246313:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.246318:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.246324:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.246338:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.299773:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:3.0:1789760781.299794:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299808:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299817:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299825:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.299830:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299846:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299865:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.299873:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b94d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.299959:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507350272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53721us (54281us total) trans 4294972901 rc 0/0 00000100:00100000:3.0:1789760781.299970:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5611 00000100:00100000:3.0:1789760781.299979:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762523707035 time 762561523728 state 0x2 ref 0 00000001:00080000:2.0:1789760781.300482:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00000100:00100000:0.0:1789760781.305613:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.305724:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350400 00000100:00100000:3.0:1789760781.305734:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350400 00000100:00100000:3.0:1789760781.305786:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762523707035 time 762561523728 state 0x0 ref 1 00000100:00100000:3.0:1789760781.305801:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762561784144 time 762561523728 state 0x1 ref 1 00000100:00100000:3.0:1789760781.305914:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762561784144 time 762621216867 state 0x0 ref 1 00000100:00100000:3.0:1789760781.305927:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c232f7d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507350400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.305974:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22863872, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.305982:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305999:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.306003:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.306010:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.306024:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.327772:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00002000:00010000:0.0:1789760781.327865:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22863872 -> 22867968 00010000:00010000:0.0:1789760781.327881:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327894:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327902:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327909:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760781.327913:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327922:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327944:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.327952:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b94dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.328056:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c232f7d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507350400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22126us (22448us total) trans 4294972902 rc 0/0 00000100:00100000:0.0:1789760781.328067:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5612 00000100:00100000:0.0:1789760781.328076:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762561784144 time 762621216867 state 0x2 ref 0 00000100:00100000:1.0:1789760781.331133:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.331210:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350528 00000100:00100000:3.0:1789760781.331219:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350528 00000100:00100000:3.0:1789760781.331272:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762561784144 time 762621216867 state 0x0 ref 1 00000100:00100000:3.0:1789760781.331286:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762621477283 time 762621216867 state 0x1 ref 1 00000100:00100000:3.0:1789760781.331314:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762621477283 time 762646617881 state 0x0 ref 1 00000100:00100000:3.0:1789760781.331327:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507350528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.331374:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22867968, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.331383:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.331399:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.331404:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.331411:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.331431:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.355250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00002000:00010000:3.0:1789760781.355427:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:3.0:1789760781.355443:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355456:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355466:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355473:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.355478:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355486:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355519:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.355537:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.355636:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507350528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24306us (24507us total) trans 4294972903 rc 0/0 00000100:00100000:3.0:1789760781.355646:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5613 00000100:00100000:3.0:1789760781.355655:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762621477283 time 762646617881 state 0x2 ref 0 00000100:00100000:1.0:1789760781.360738:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.362532:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350656 00000100:00100000:3.0:1789760781.362547:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350656 00000100:00100000:3.0:1789760781.362604:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762621477283 time 762646617881 state 0x0 ref 1 00000100:00100000:3.0:1789760781.362618:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762646878297 time 762646617881 state 0x1 ref 1 00000100:00100000:3.0:1789760781.362652:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762646878297 time 762677956001 state 0x0 ref 1 00000100:00100000:3.0:1789760781.362666:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507350656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.362741:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22872064, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.362750:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.362765:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.362770:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.362776:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.362789:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.399900:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:2.0:1789760781.399920:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399937:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399947:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399953:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.399958:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399966:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399991:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.399998:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b94eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.400087:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507350656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37417us (39353us total) trans 4294972904 rc 0/0 00000100:00100000:2.0:1789760781.400097:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5614 00000100:00100000:2.0:1789760781.400108:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762646878297 time 762677956001 state 0x2 ref 0 00000001:00080000:1.0:1789760781.401966:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00000100:00100000:0.0:1789760781.405815:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.406139:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350784 00000100:00100000:3.0:1789760781.406148:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350784 00000100:00100000:3.0:1789760781.406202:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762646878297 time 762677956001 state 0x0 ref 1 00000100:00100000:3.0:1789760781.406217:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762678216417 time 762677956001 state 0x1 ref 1 00000100:00100000:3.0:1789760781.406282:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762678216417 time 762721584912 state 0x0 ref 1 00000100:00100000:3.0:1789760781.406295:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507350784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760781.406341:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22876160, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:3.0:1789760781.406350:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.406366:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760781.406371:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.406377:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.406390:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.429542:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:1.0:1789760781.429675:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:1.0:1789760781.429689:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429749:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429760:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429767:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.429772:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429781:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429799:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.429807:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b94f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.429891:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507350784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23594us (24082us total) trans 4294972905 rc 0/0 00000100:00100000:1.0:1789760781.429900:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5615 00000100:00100000:1.0:1789760781.429909:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762678216417 time 762721584912 state 0x2 ref 0 00000100:00100000:0.0:1789760781.436238:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760781.436374:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507350912 00000100:00100000:3.0:1789760781.436385:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507350912 00000100:00100000:3.0:1789760781.436440:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762678216417 time 762721584912 state 0x0 ref 1 00000100:00100000:3.0:1789760781.436478:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762721845328 time 762721584912 state 0x1 ref 1 00000100:00100000:3.0:1789760781.436524:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762721845328 time 762751826966 state 0x0 ref 1 00000100:00100000:3.0:1789760781.436537:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507350912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760781.436670:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22880256, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:0.0:1789760781.436679:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.436719:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760781.436724:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.436731:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.436748:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.463716:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00002000:00010000:1.0:1789760781.463812:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:1.0:1789760781.463827:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463846:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463856:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463864:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.463868:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463878:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463897:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.463905:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b94f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.464038:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507350912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27498us (27804us total) trans 4294972906 rc 0/0 00000100:00100000:1.0:1789760781.464049:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5616 00000100:00100000:1.0:1789760781.464059:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762721845328 time 762751826966 state 0x2 ref 0 00000100:00100000:0.0:1789760781.477648:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760781.478039:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351040 00000100:00100000:2.0:1789760781.478049:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351040 00000100:00100000:2.0:1789760781.478319:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762721845328 time 762751826966 state 0x0 ref 1 00000100:00100000:2.0:1789760781.478338:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762752087382 time 762751826966 state 0x1 ref 1 00000100:00100000:2.0:1789760781.478369:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762752087382 time 762793672239 state 0x0 ref 1 00000100:00100000:2.0:1789760781.478382:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507351040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.478431:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22884352, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.478440:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.478461:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.478466:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.478473:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.478500:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.514499:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:2.0:1789760781.514635:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:2.0:1789760781.514656:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514669:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514678:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514685:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.514716:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514727:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514747:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.514756:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9500 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.514841:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507351040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36456us (37198us total) trans 4294972907 rc 0/0 00000100:00100000:2.0:1789760781.514852:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5617 00000100:00100000:2.0:1789760781.514861:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762752087382 time 762793672239 state 0x2 ref 0 00000100:00100000:2.0:1789760781.519370:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.519458:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351168 00000100:00100000:1.0:1789760781.519468:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351168 00000100:00100000:1.0:1789760781.519531:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762752087382 time 762793672239 state 0x0 ref 1 00000100:00100000:1.0:1789760781.519547:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762793932655 time 762793672239 state 0x1 ref 1 00000100:00100000:1.0:1789760781.519576:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762793932655 time 762834879329 state 0x0 ref 1 00000100:00100000:1.0:1789760781.519589:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760781.519638:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22888448, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760781.519648:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.519665:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760781.519670:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.519677:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.519727:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.534178:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538292480:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760781.534282:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760781.534403:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292480 00000100:00100000:0.0:1789760781.534412:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292480 00000100:00100000:0.0:1789760781.535224:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1789760781.535236:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538292480:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760781.535275:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760781.535340:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538292480:12345-0@lo:41:osp-pre-0-1.0 Request processed in 107us (1058us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.535347:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789760781.535504:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c240fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538292480:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760781.550316:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00002000:00010000:2.0:1789760781.550724:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:2.0:1789760781.550738:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550751:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550761:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550770:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.550775:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550785:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550804:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550812:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9507 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.550902:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31311us (31536us total) trans 4294972908 rc 0/0 00000100:00100000:2.0:1789760781.550912:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5618 00000100:00100000:2.0:1789760781.550920:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762793932655 time 762834879329 state 0x2 ref 0 00000100:00100000:1.0:1789760781.559333:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760781.559422:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351296 00000100:00100000:0.0:1789760781.559433:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351296 00000100:00100000:0.0:1789760781.559520:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762793932655 time 762834879329 state 0x0 ref 1 00000100:00100000:0.0:1789760781.559536:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762835139745 time 762834879329 state 0x1 ref 1 00000100:00100000:0.0:1789760781.559561:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762835139745 time 762874864556 state 0x0 ref 1 00000100:00100000:0.0:1789760781.559574:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760781.559621:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22892544, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:0.0:1789760781.559630:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.559646:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760781.559651:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.559658:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.559671:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.610829:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00002000:00010000:1.0:1789760781.610968:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:1.0:1789760781.610982:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.610995:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.611004:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.611012:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.611017:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.611027:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.611045:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.611053:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b950e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.611143:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51567us (51816us total) trans 4294972909 rc 0/0 00000100:00100000:1.0:1789760781.611153:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5619 00000100:00100000:1.0:1789760781.611163:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762835139745 time 762874864556 state 0x2 ref 0 00000100:00100000:0.0:1789760781.621831:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760781.621900:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351424 00000100:00100000:2.0:1789760781.621911:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351424 00000100:00100000:2.0:1789760781.621974:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762835139745 time 762874864556 state 0x0 ref 1 00000100:00100000:2.0:1789760781.621989:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762875124972 time 762874864556 state 0x1 ref 1 00000100:00100000:2.0:1789760781.622019:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762875124972 time 762937322767 state 0x0 ref 1 00000100:00100000:2.0:1789760781.622033:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507351424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.622087:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22896640, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.622097:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622113:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.622117:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.622123:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622138:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.660823:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00002000:00010000:1.0:1789760781.660962:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:1.0:1789760781.660977:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.660996:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.661005:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.661011:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.661016:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.661024:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.661042:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.661049:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9515 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.661141:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507351424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39105us (39314us total) trans 4294972910 rc 0/0 00000100:00100000:1.0:1789760781.661153:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5620 00000100:00100000:1.0:1789760781.661161:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762875124972 time 762937322767 state 0x2 ref 0 00000100:00100000:2.0:1789760781.666516:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.666777:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351552 00000100:00100000:1.0:1789760781.666788:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351552 00000100:00100000:2.0:1789760781.667113:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762875124972 time 762937322767 state 0x0 ref 1 00000100:00100000:2.0:1789760781.667129:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762937583183 time 762937322767 state 0x1 ref 1 00000100:00100000:2.0:1789760781.667156:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762937583183 time 762982460353 state 0x0 ref 1 00000100:00100000:2.0:1789760781.667168:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507351552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.667313:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22900736, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.667323:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.667338:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760781.667342:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.667349:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.667369:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.709825:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22900736 -> 22904832 00000001:00080000:0.0:1789760781.709921:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00010000:00010000:2.0:1789760781.710038:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710052:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710061:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710068:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.710073:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710082:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710104:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.710112:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b951c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.710569:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507351552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43116us (43775us total) trans 4294972911 rc 0/0 00000100:00100000:2.0:1789760781.710648:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5621 00000100:00100000:2.0:1789760781.710659:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762937583183 time 762982460353 state 0x2 ref 0 00000100:00100000:0.0:1789760781.716688:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.716853:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351680 00000100:00100000:1.0:1789760781.716862:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351680 00000100:00100000:1.0:1789760781.716914:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762937583183 time 762982460353 state 0x0 ref 1 00000100:00100000:1.0:1789760781.716928:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762982720769 time 762982460353 state 0x1 ref 1 00000100:00100000:1.0:1789760781.716966:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762982720769 time 763032269496 state 0x0 ref 1 00000100:00100000:1.0:1789760781.717082:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760781.717130:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22904832, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760781.717139:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.717155:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760781.717160:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.717166:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.717179:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.772977:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:3.0:1789760781.772997:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.773009:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.773018:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.773026:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760781.773031:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.773039:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.773060:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.773067:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00010000:00010000:3.0:1789760781.773068:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9523 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.773153:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56070us (56578us total) trans 4294972912 rc 0/0 00000100:00100000:3.0:1789760781.773163:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5622 00000100:00100000:3.0:1789760781.773173:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762982720769 time 763032269496 state 0x2 ref 0 00000100:00100000:0.0:1789760781.778803:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.779651:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351808 00000100:00100000:1.0:1789760781.779661:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351808 00000100:00100000:1.0:1789760781.779806:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 762982720769 time 763032269496 state 0x0 ref 1 00000100:00100000:1.0:1789760781.779822:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763032529912 time 763032269496 state 0x1 ref 1 00000100:00100000:1.0:1789760781.780038:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763032529912 time 763095339710 state 0x0 ref 1 00000100:00100000:1.0:1789760781.780051:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507351808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760781.780100:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22908928, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760781.780109:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.780124:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760781.780128:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.780134:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.780148:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.822975:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:1.0:1789760781.822996:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823020:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823029:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823036:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760781.823042:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823051:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823070:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.823077:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b952a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.823170:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507351808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43117us (44368us total) trans 4294972913 rc 0/0 00000100:00100000:1.0:1789760781.823180:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5623 00000100:00100000:1.0:1789760781.823189:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763032529912 time 763095339710 state 0x2 ref 0 00000001:00080000:2.0:1789760781.823864:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00000100:00100000:2.0:1789760781.826413:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760781.826521:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507351936 00000100:00100000:0.0:1789760781.826532:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507351936 00000100:00100000:0.0:1789760781.826594:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763032529912 time 763095339710 state 0x0 ref 1 00000100:00100000:0.0:1789760781.826609:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763095600126 time 763095339710 state 0x1 ref 1 00000100:00100000:0.0:1789760781.826639:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763095600126 time 763141942469 state 0x0 ref 1 00000100:00100000:0.0:1789760781.826650:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507351936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760781.826717:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22913024, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:0.0:1789760781.826728:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.826743:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760781.826749:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.826755:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.826770:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760781.849468:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22913024 -> 22917120 00000001:00080000:2.0:1789760781.849496:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00010000:00010000:0.0:1789760781.849499:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849511:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849521:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849595:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760781.849601:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849610:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849631:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.849639:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9531 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.849861:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507351936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23207us (23452us total) trans 4294972914 rc 0/0 00000100:00100000:0.0:1789760781.849872:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5624 00000100:00100000:0.0:1789760781.849882:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763095600126 time 763141942469 state 0x2 ref 0 00000100:00100000:3.0:1789760781.853634:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760781.853844:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352064 00000100:00100000:1.0:1789760781.853854:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352064 00000100:00100000:1.0:1789760781.854021:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763095600126 time 763141942469 state 0x0 ref 1 00000100:00100000:1.0:1789760781.854036:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763142202885 time 763141942469 state 0x1 ref 1 00000100:00100000:1.0:1789760781.854066:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763142202885 time 763169370133 state 0x0 ref 1 00000100:00100000:1.0:1789760781.854079:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507352064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760781.854298:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22917120, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760781.854306:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.854320:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760781.854323:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.854328:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.854338:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.904283:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00002000:00010000:2.0:1789760781.905011:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22917120 -> 22921216 00010000:00010000:2.0:1789760781.905030:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905044:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905054:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905073:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.905078:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905087:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905113:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.905121:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9538 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.905397:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507352064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51314us (51766us total) trans 4294972915 rc 0/0 00000100:00100000:2.0:1789760781.905408:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5625 00000100:00100000:2.0:1789760781.905419:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763142202885 time 763169370133 state 0x2 ref 0 00000100:00100000:1.0:1789760781.913550:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760781.913729:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352192 00000100:00100000:2.0:1789760781.913746:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352192 00000100:00100000:2.0:1789760781.913798:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763142202885 time 763169370133 state 0x0 ref 1 00000100:00100000:2.0:1789760781.913818:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763169630549 time 763169370133 state 0x1 ref 1 00000100:00100000:2.0:1789760781.914179:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763169630549 time 763229481199 state 0x0 ref 1 00000100:00100000:2.0:1789760781.914193:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507352192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760781.914240:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22921216, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:2.0:1789760781.914249:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.914265:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760781.914271:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.914278:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.914292:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.971788:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00002000:00010000:2.0:1789760781.971931:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22921216 -> 22925312 00010000:00010000:2.0:1789760781.971946:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.971960:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.971968:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.971976:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760781.971981:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.971991:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.972010:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.972023:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b953f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.972123:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507352192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57927us (58579us total) trans 4294972916 rc 0/0 00000100:00100000:2.0:1789760781.972133:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5626 00000100:00100000:2.0:1789760781.972143:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763169630549 time 763229481199 state 0x2 ref 0 00000100:00100000:1.0:1789760781.979928:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760781.980180:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352320 00000100:00100000:0.0:1789760781.980190:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352320 00000100:00100000:0.0:1789760781.980244:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763169630549 time 763229481199 state 0x0 ref 1 00000100:00100000:0.0:1789760781.980263:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763229741615 time 763229481199 state 0x1 ref 1 00000100:00100000:0.0:1789760781.980295:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763229741615 time 763295597596 state 0x0 ref 1 00000100:00100000:0.0:1789760781.980307:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507352320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760781.980354:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22925312, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:0.0:1789760781.980362:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.980378:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760781.980382:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.980388:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.980403:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.016412:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00002000:00010000:0.0:1789760782.016586:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22925312 -> 22929408 00010000:00010000:0.0:1789760782.016603:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016615:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016624:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016637:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.016642:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016651:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016670:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.016679:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9546 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.016858:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507352320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36549us (36935us total) trans 4294972917 rc 0/0 00000100:00100000:0.0:1789760782.016869:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5627 00000100:00100000:0.0:1789760782.016879:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763229741615 time 763295597596 state 0x2 ref 0 00000100:00100000:2.0:1789760782.020898:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760782.021056:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352448 00000100:00100000:1.0:1789760782.021066:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352448 00000100:00100000:1.0:1789760782.021119:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763229741615 time 763295597596 state 0x0 ref 1 00000100:00100000:1.0:1789760782.021136:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763295858012 time 763295597596 state 0x1 ref 1 00000100:00100000:1.0:1789760782.021210:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763295858012 time 763336511661 state 0x0 ref 1 00000100:00100000:1.0:1789760782.021223:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760782.021274:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22929408, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xb007 00010000:00010000:1.0:1789760782.021283:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.021304:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760782.021309:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.021316:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.021330:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.048905:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22929408 -> 22933504 00002000:00010000:0.0:1789760782.048915:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:0.0:1789760782.048918:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:0.0:1789760782.048941:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.048954:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.048964:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.048972:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.048977:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.048987:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.049007:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.049015:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b954d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.049110:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507352448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27884us (28216us total) trans 4294972918 rc 0/0 00000100:00100000:0.0:1789760782.049121:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5628 00000100:00100000:0.0:1789760782.049130:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763295858012 time 763336511661 state 0x2 ref 0 00000001:00080000:3.0:1789760782.049222:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00000100:00100000:3.0:1789760782.052838:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.053027:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352576 00000100:00100000:2.0:1789760782.053036:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352576 00000100:00100000:2.0:1789760782.053092:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763295858012 time 763336511661 state 0x0 ref 1 00000100:00100000:2.0:1789760782.053107:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763336772077 time 763336511661 state 0x1 ref 1 00000100:00100000:2.0:1789760782.053132:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763336772077 time 763368435206 state 0x0 ref 1 00000100:00100000:2.0:1789760782.053144:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.053191:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22933504, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.053199:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.053214:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760782.053218:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.053224:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.053238:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.081589:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:2.0:1789760782.081609:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081622:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081632:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081640:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.081644:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081653:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081672:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.081680:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9554 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.081804:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507352576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28657us (28970us total) trans 4294972919 rc 0/0 00000100:00100000:2.0:1789760782.081813:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5629 00000100:00100000:2.0:1789760782.081823:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763336772077 time 763368435206 state 0x2 ref 0 00000001:00080000:3.0:1789760782.082056:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00000100:00100000:2.0:1789760782.089886:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.089978:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352704 00000100:00100000:0.0:1789760782.089989:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352704 00000100:00100000:0.0:1789760782.090048:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763336772077 time 763368435206 state 0x0 ref 1 00000100:00100000:0.0:1789760782.090064:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763368695622 time 763368435206 state 0x1 ref 1 00000100:00100000:0.0:1789760782.090097:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763368695622 time 763405399949 state 0x0 ref 1 00000100:00100000:0.0:1789760782.090110:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.090158:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22937600, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.090166:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.090183:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.090187:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.090194:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.090208:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.129320:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:0.0:1789760782.129339:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129352:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129361:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129368:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.129374:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129383:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129401:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.129408:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b955b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.129511:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507352704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39398us (39628us total) trans 4294972920 rc 0/0 00000001:00080000:3.0:1789760782.129617:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00000100:00100000:0.0:1789760782.129664:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5630 00000100:00100000:0.0:1789760782.129675:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763368695622 time 763405399949 state 0x2 ref 0 00000100:00100000:0.0:1789760782.136123:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760782.136601:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352832 00000100:00100000:1.0:1789760782.136612:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352832 00000100:00100000:1.0:1789760782.136666:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763368695622 time 763405399949 state 0x0 ref 1 00000100:00100000:1.0:1789760782.136681:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763405660365 time 763405399949 state 0x1 ref 1 00000100:00100000:3.0:1789760782.136852:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763405660365 time 763452152795 state 0x0 ref 1 00000100:00100000:3.0:1789760782.136866:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760782.136910:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22941696, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:3.0:1789760782.136917:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.137773:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760782.137778:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.137785:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.137800:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.207360:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:2.0:1789760782.207447:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:2.0:1789760782.207463:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207474:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207494:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207501:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.207505:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207513:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207534:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.207546:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9562 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.207643:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70774us (71532us total) trans 4294972921 rc 0/0 00000100:00100000:2.0:1789760782.207652:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5631 00000100:00100000:2.0:1789760782.207663:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763405660365 time 763452152795 state 0x2 ref 0 00000100:00100000:2.0:1789760782.213816:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760782.213961:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507352960 00000100:00100000:1.0:1789760782.213971:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507352960 00000100:00100000:1.0:1789760782.214024:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763405660365 time 763452152795 state 0x0 ref 1 00000100:00100000:1.0:1789760782.214039:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763452413211 time 763452152795 state 0x1 ref 1 00000100:00100000:1.0:1789760782.214070:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763452413211 time 763529373593 state 0x0 ref 1 00000100:00100000:1.0:1789760782.214083:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760782.214129:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22945792, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:1.0:1789760782.214138:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.214159:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760782.214164:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.214170:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.214183:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.236591:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760782.264735:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00002000:00010000:0.0:1789760782.264792:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:0.0:1789760782.264807:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264819:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264833:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264841:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.264846:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264856:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264877:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.264884:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9569 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.265195:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507352960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51108us (51382us total) trans 4294972922 rc 0/0 00000100:00100000:0.0:1789760782.265206:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5632 00000100:00100000:0.0:1789760782.265216:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763452413211 time 763529373593 state 0x2 ref 0 00000100:00100000:1.0:1789760782.271502:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.271736:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353088 00000100:00100000:0.0:1789760782.271746:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353088 00000100:00100000:0.0:1789760782.271803:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763452413211 time 763529373593 state 0x0 ref 1 00000100:00100000:0.0:1789760782.271819:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763529634009 time 763529373593 state 0x1 ref 1 00000100:00100000:0.0:1789760782.271842:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763529634009 time 763587144857 state 0x0 ref 1 00000100:00100000:0.0:1789760782.271855:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.271903:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22949888, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.271912:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.271927:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.271932:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.271938:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.271953:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.306164:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00002000:00010000:0.0:1789760782.306358:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:0.0:1789760782.306374:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306387:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306396:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306404:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.306409:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306419:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306438:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.306445:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9570 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.306883:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35022us (35394us total) trans 4294972923 rc 0/0 00000100:00100000:0.0:1789760782.306894:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5633 00000100:00100000:0.0:1789760782.306903:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763529634009 time 763587144857 state 0x2 ref 0 00000100:00100000:2.0:1789760782.312338:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760782.312410:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353216 00000100:00100000:3.0:1789760782.312419:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353216 00000100:00100000:3.0:1789760782.312470:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763529634009 time 763587144857 state 0x0 ref 1 00000100:00100000:3.0:1789760782.312493:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763587405273 time 763587144857 state 0x1 ref 1 00000100:00100000:3.0:1789760782.312517:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763587405273 time 763627820534 state 0x0 ref 1 00000100:00100000:3.0:1789760782.312535:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760782.312576:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22953984, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:3.0:1789760782.312584:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312599:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760782.312604:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.312610:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312624:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.339441:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00002000:00010000:2.0:1789760782.339598:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:2.0:1789760782.339614:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339628:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339638:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339645:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.339650:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339660:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339681:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339689:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9577 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.339837:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27298us (27503us total) trans 4294972924 rc 0/0 00000100:00100000:2.0:1789760782.339847:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5634 00000100:00100000:2.0:1789760782.339856:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763587405273 time 763627820534 state 0x2 ref 0 00000100:00100000:1.0:1789760782.343633:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.343717:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353344 00000100:00100000:0.0:1789760782.343729:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353344 00000100:00100000:0.0:1789760782.343788:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763587405273 time 763627820534 state 0x0 ref 1 00000100:00100000:0.0:1789760782.343802:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763628080950 time 763627820534 state 0x1 ref 1 00000100:00100000:0.0:1789760782.343823:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763628080950 time 763659126809 state 0x0 ref 1 00000100:00100000:0.0:1789760782.343836:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.343887:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22958080, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.343897:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.343913:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.343918:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.343924:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.343937:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.375869:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00002000:00010000:2.0:1789760782.375981:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:2.0:1789760782.375998:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376010:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376019:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376026:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.376032:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376046:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376067:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.376074:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b957e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.376168:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32331us (32540us total) trans 4294972925 rc 0/0 00000100:00100000:2.0:1789760782.376179:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5635 00000100:00100000:2.0:1789760782.376189:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763628080950 time 763659126809 state 0x2 ref 0 00000100:00100000:1.0:1789760782.379858:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.379962:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353472 00000100:00100000:0.0:1789760782.379972:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353472 00000100:00100000:0.0:1789760782.380028:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763628080950 time 763659126809 state 0x0 ref 1 00000100:00100000:0.0:1789760782.380043:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763659387225 time 763659126809 state 0x1 ref 1 00000100:00100000:0.0:1789760782.380075:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763659387225 time 763695378593 state 0x0 ref 1 00000100:00100000:0.0:1789760782.380087:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507353472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.380135:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22962176, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.380144:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.380160:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.380165:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.380172:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.380186:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.413727:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:0.0:1789760782.413827:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:0.0:1789760782.413842:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413855:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413864:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413871:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.413875:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413884:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413904:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.413918:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9585 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.414019:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507353472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33928us (34166us total) trans 4294972926 rc 0/0 00000100:00100000:0.0:1789760782.414030:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5636 00000100:00100000:0.0:1789760782.414039:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763659387225 time 763695378593 state 0x2 ref 0 00000100:00100000:0.0:1789760782.418090:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.418194:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353600 00000100:00100000:2.0:1789760782.418204:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353600 00000100:00100000:2.0:1789760782.418260:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763659387225 time 763695378593 state 0x0 ref 1 00000100:00100000:2.0:1789760782.418274:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763695639009 time 763695378593 state 0x1 ref 1 00000100:00100000:2.0:1789760782.418301:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763695639009 time 763733605023 state 0x0 ref 1 00000100:00100000:2.0:1789760782.418313:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507353600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.418359:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22966272, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.418368:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.418384:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760782.418389:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.418395:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.418410:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.448993:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00002000:00010000:2.0:1789760782.449091:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:2.0:1789760782.449106:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449118:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449127:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449134:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.449138:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449147:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449170:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.449177:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b958c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.449270:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507353600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30954us (31186us total) trans 4294972927 rc 0/0 00000100:00100000:2.0:1789760782.449280:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5637 00000100:00100000:2.0:1789760782.449290:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763695639009 time 763733605023 state 0x2 ref 0 00000100:00100000:2.0:1789760782.453190:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.453270:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353728 00000100:00100000:0.0:1789760782.453279:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353728 00000100:00100000:0.0:1789760782.453333:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763695639009 time 763733605023 state 0x0 ref 1 00000100:00100000:0.0:1789760782.453349:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763733865439 time 763733605023 state 0x1 ref 1 00000100:00100000:0.0:1789760782.453380:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763733865439 time 763768683547 state 0x0 ref 1 00000100:00100000:0.0:1789760782.453393:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.453443:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22970368, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.453451:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.453467:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.453472:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.453491:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.453507:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.485718:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00002000:00010000:0.0:1789760782.485829:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:0.0:1789760782.485844:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485857:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485866:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485874:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.485879:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485888:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485908:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.485916:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9593 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.486121:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32723us (32933us total) trans 4294972928 rc 0/0 00000100:00100000:0.0:1789760782.486132:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5638 00000100:00100000:0.0:1789760782.486142:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763733865439 time 763768683547 state 0x2 ref 0 00000100:00100000:0.0:1789760782.489955:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.490188:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353856 00000100:00100000:2.0:1789760782.490198:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353856 00000100:00100000:2.0:1789760782.490250:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763733865439 time 763768683547 state 0x0 ref 1 00000100:00100000:2.0:1789760782.490265:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763768943963 time 763768683547 state 0x1 ref 1 00000100:00100000:2.0:1789760782.490296:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763768943963 time 763805599496 state 0x0 ref 1 00000100:00100000:2.0:1789760782.490309:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507353856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.490357:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22974464, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.490366:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.490389:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760782.490393:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.490399:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.490413:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.522763:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00002000:00010000:0.0:1789760782.522876:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22974464 -> 22978560 00010000:00010000:0.0:1789760782.522890:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.522904:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.522913:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.522920:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.522988:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.523000:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.523021:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.523035:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b959a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.523158:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507353856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32847us (33208us total) trans 4294972929 rc 0/0 00000100:00100000:0.0:1789760782.523169:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5639 00000100:00100000:0.0:1789760782.523179:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763768943963 time 763805599496 state 0x2 ref 0 00000100:00100000:2.0:1789760782.526305:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760782.526408:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507353984 00000100:00100000:3.0:1789760782.526417:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507353984 00000100:00100000:3.0:1789760782.526472:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763768943963 time 763805599496 state 0x0 ref 1 00000100:00100000:3.0:1789760782.526499:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763805859912 time 763805599496 state 0x1 ref 1 00000100:00100000:3.0:1789760782.526620:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763805859912 time 763841921820 state 0x0 ref 1 00000100:00100000:3.0:1789760782.526633:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760782.526680:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22978560, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:3.0:1789760782.526689:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.526725:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760782.526730:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.526736:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.526750:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.565322:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00002000:00010000:3.0:1789760782.565413:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:3.0:1789760782.565429:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565443:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565459:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565466:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760782.565471:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565489:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565507:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.565513:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.565619:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507353984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38982us (39317us total) trans 4294972930 rc 0/0 00000100:00100000:3.0:1789760782.565629:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5640 00000100:00100000:3.0:1789760782.565638:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763805859912 time 763841921820 state 0x2 ref 0 00000100:00100000:2.0:1789760782.569231:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.569320:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354112 00000100:00100000:0.0:1789760782.569329:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354112 00000100:00100000:0.0:1789760782.569384:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763805859912 time 763841921820 state 0x0 ref 1 00000100:00100000:0.0:1789760782.569399:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763842182236 time 763841921820 state 0x1 ref 1 00000100:00100000:0.0:1789760782.569427:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763842182236 time 763884730889 state 0x0 ref 1 00000100:00100000:0.0:1789760782.569438:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507354112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.569499:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22982656, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.569507:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.569531:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760782.569536:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.569543:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.569558:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.610550:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22982656 -> 22986752 00010000:00010000:2.0:1789760782.610571:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610583:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610592:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610599:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.610605:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610614:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610639:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.610648:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b95a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.610924:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507354112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41480us (41697us total) trans 4294972931 rc 0/0 00000100:00100000:2.0:1789760782.610935:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5641 00000100:00100000:2.0:1789760782.610946:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763842182236 time 763884730889 state 0x2 ref 0 00000001:00080000:3.0:1789760782.611310:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00000100:00100000:0.0:1789760782.614026:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.614240:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354240 00000100:00100000:2.0:1789760782.614251:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354240 00000100:00100000:2.0:1789760782.614306:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763842182236 time 763884730889 state 0x0 ref 1 00000100:00100000:2.0:1789760782.614321:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763884991305 time 763884730889 state 0x1 ref 1 00000100:00100000:2.0:1789760782.614375:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763884991305 time 763929677849 state 0x0 ref 1 00000100:00100000:2.0:1789760782.614388:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507354240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.614436:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22986752, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.614444:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.614460:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760782.614466:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.614472:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.614497:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.639286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00002000:00010000:2.0:1789760782.639546:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:2.0:1789760782.639562:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639575:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639584:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639590:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.639594:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639602:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639620:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.639632:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.639762:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507354240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25366us (25739us total) trans 4294972932 rc 0/0 00000100:00100000:2.0:1789760782.639773:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5642 00000100:00100000:2.0:1789760782.639783:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763884991305 time 763929677849 state 0x2 ref 0 00000100:00100000:2.0:1789760782.646054:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760782.646475:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354368 00000100:00100000:3.0:1789760782.646497:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354368 00000100:00100000:3.0:1789760782.646678:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763884991305 time 763929677849 state 0x0 ref 1 00000100:00100000:3.0:1789760782.646806:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763929938265 time 763929677849 state 0x1 ref 1 00000100:00100000:3.0:1789760782.646847:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763929938265 time 763962150638 state 0x0 ref 1 00000100:00100000:3.0:1789760782.646862:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507354368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760782.646910:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22990848, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:3.0:1789760782.646919:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.646936:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760782.646940:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.646946:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.646960:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.685806:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00002000:00010000:0.0:1789760782.685857:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22990848 -> 22994944 00010000:00010000:0.0:1789760782.685873:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685886:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685896:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685901:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.685905:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685911:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685927:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685932:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b95b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.686011:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507354368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39148us (39965us total) trans 4294972933 rc 0/0 00000100:00100000:0.0:1789760782.686022:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5643 00000100:00100000:0.0:1789760782.686031:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763929938265 time 763962150638 state 0x2 ref 0 00000100:00100000:3.0:1789760782.690054:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.690391:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354496 00000100:00100000:2.0:1789760782.690401:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354496 00000100:00100000:2.0:1789760782.690461:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763929938265 time 763962150638 state 0x0 ref 1 00000100:00100000:2.0:1789760782.690476:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763962411054 time 763962150638 state 0x1 ref 1 00000100:00100000:2.0:1789760782.690595:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763962411054 time 764005889256 state 0x0 ref 1 00000100:00100000:2.0:1789760782.690609:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507354496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.690658:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22994944, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.690668:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.690682:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760782.690687:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.690722:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.690738:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.714970:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00002000:00010000:0.0:1789760782.715055:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:0.0:1789760782.715070:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715083:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715092:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715105:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.715110:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715119:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715140:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.715163:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b95bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.715249:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507354496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24639us (25199us total) trans 4294972934 rc 0/0 00000100:00100000:0.0:1789760782.715258:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5644 00000100:00100000:0.0:1789760782.715268:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763962411054 time 764005889256 state 0x2 ref 0 00000100:00100000:1.0:1789760782.721689:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.722023:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354624 00000100:00100000:2.0:1789760782.722033:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354624 00000100:00100000:2.0:1789760782.722091:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 763962411054 time 764005889256 state 0x0 ref 1 00000100:00100000:2.0:1789760782.722108:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764006149672 time 764005889256 state 0x1 ref 1 00000100:00100000:1.0:1789760782.722163:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764006149672 time 764037464295 state 0x0 ref 1 00000100:00100000:1.0:1789760782.722176:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507354624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760782.722225:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22999040, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:1.0:1789760782.722234:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.722249:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760782.722254:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.722260:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.722274:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.745110:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00002000:00010000:0.0:1789760782.745188:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:0.0:1789760782.745217:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745231:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745241:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745250:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.745256:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745380:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745474:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.745494:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b95c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.745585:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507354624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23406us (23898us total) trans 4294972935 rc 0/0 00000100:00100000:0.0:1789760782.745599:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5645 00000100:00100000:0.0:1789760782.745610:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764006149672 time 764037464295 state 0x2 ref 0 00000100:00100000:2.0:1789760782.749808:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.749917:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354752 00000100:00100000:0.0:1789760782.749927:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354752 00000100:00100000:0.0:1789760782.749983:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764006149672 time 764037464295 state 0x0 ref 1 00000100:00100000:0.0:1789760782.750008:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764037724711 time 764037464295 state 0x1 ref 1 00000100:00100000:0.0:1789760782.750041:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764037724711 time 764065343688 state 0x0 ref 1 00000100:00100000:0.0:1789760782.750053:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507354752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.750101:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23003136, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.750110:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.750127:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.750133:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.750139:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.750154:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.775064:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:0.0:1789760782.775085:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775099:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775108:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775116:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.775120:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775129:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775149:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.775156:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b95cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.775242:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507354752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25186us (25440us total) trans 4294972936 rc 0/0 00000100:00100000:0.0:1789760782.775253:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5646 00000100:00100000:0.0:1789760782.775262:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764037724711 time 764065343688 state 0x2 ref 0 00000001:00080000:3.0:1789760782.775335:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00000100:00100000:3.0:1789760782.778612:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.778735:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507354880 00000100:00100000:0.0:1789760782.778747:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507354880 00000100:00100000:0.0:1789760782.778807:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764037724711 time 764065343688 state 0x0 ref 1 00000100:00100000:0.0:1789760782.778821:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764065604104 time 764065343688 state 0x1 ref 1 00000100:00100000:0.0:1789760782.778853:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764065604104 time 764094155753 state 0x0 ref 1 00000100:00100000:0.0:1789760782.778874:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507354880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.778921:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23007232, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.778930:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.778946:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.778951:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.778957:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.778971:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.813601:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00002000:00010000:0.0:1789760782.814030:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23007232 -> 23011328 00010000:00010000:0.0:1789760782.814045:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814057:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814066:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814074:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.814079:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814087:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814106:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814113:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.814259:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507354880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35382us (35653us total) trans 4294972937 rc 0/0 00000100:00100000:0.0:1789760782.814274:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5647 00000100:00100000:0.0:1789760782.814284:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764065604104 time 764094155753 state 0x2 ref 0 00000100:00100000:1.0:1789760782.821339:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.821439:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355008 00000100:00100000:0.0:1789760782.821449:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355008 00000100:00100000:0.0:1789760782.821511:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764065604104 time 764094155753 state 0x0 ref 1 00000100:00100000:0.0:1789760782.821639:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764094416169 time 764094155753 state 0x1 ref 1 00000100:00100000:0.0:1789760782.821670:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764094416169 time 764136973216 state 0x0 ref 1 00000100:00100000:0.0:1789760782.821683:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507355008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.821749:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23011328, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.821759:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.821774:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.821779:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.821785:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.821907:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.848053:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00002000:00010000:2.0:1789760782.848123:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:2.0:1789760782.848137:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848150:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848160:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848168:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760782.848173:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848187:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848209:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.848218:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b95d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.848313:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507355008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26628us (26980us total) trans 4294972938 rc 0/0 00000100:00100000:2.0:1789760782.848323:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5648 00000100:00100000:2.0:1789760782.848334:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764094416169 time 764136973216 state 0x2 ref 0 00000100:00100000:2.0:1789760782.853255:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.853343:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355136 00000100:00100000:0.0:1789760782.853353:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355136 00000100:00100000:0.0:1789760782.853407:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764094416169 time 764136973216 state 0x0 ref 1 00000100:00100000:0.0:1789760782.853422:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764137233632 time 764136973216 state 0x1 ref 1 00000100:00100000:0.0:1789760782.853492:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764137233632 time 764168782190 state 0x0 ref 1 00000100:00100000:0.0:1789760782.853505:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.853561:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23015424, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.853568:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.853580:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.853583:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.853588:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.853600:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.883771:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00002000:00010000:0.0:1789760782.883894:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:0.0:1789760782.883909:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883923:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883932:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883939:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.883944:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883952:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883969:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.883976:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.884293:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30784us (31039us total) trans 4294972939 rc 0/0 00000100:00100000:0.0:1789760782.884305:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5649 00000100:00100000:0.0:1789760782.884316:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764137233632 time 764168782190 state 0x2 ref 0 00000100:00100000:2.0:1789760782.892790:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.892897:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355264 00000100:00100000:0.0:1789760782.892965:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355264 00000100:00100000:0.0:1789760782.893154:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764137233632 time 764168782190 state 0x0 ref 1 00000100:00100000:0.0:1789760782.893170:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764169042606 time 764168782190 state 0x1 ref 1 00000100:00100000:0.0:1789760782.893245:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764169042606 time 764208547225 state 0x0 ref 1 00000100:00100000:0.0:1789760782.893261:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507355264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.893309:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23019520, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.893317:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.893334:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.893338:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.893344:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.893358:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.926688:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00002000:00010000:0.0:1789760782.926830:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:0.0:1789760782.926845:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926861:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926870:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926876:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.926881:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926891:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926915:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.926923:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.927057:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507355264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33793us (34273us total) trans 4294972940 rc 0/0 00000100:00100000:0.0:1789760782.927068:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5650 00000100:00100000:0.0:1789760782.927077:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764169042606 time 764208547225 state 0x2 ref 0 00000100:00100000:0.0:1789760782.936943:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760782.937794:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355392 00000100:00100000:2.0:1789760782.937808:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355392 00000100:00100000:2.0:1789760782.937878:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764169042606 time 764208547225 state 0x0 ref 1 00000100:00100000:2.0:1789760782.937894:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764208807641 time 764208547225 state 0x1 ref 1 00000100:00100000:2.0:1789760782.937927:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764208807641 time 764253230019 state 0x0 ref 1 00000100:00100000:2.0:1789760782.937943:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507355392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760782.938032:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23023616, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:2.0:1789760782.938042:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.938056:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760782.938061:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.938067:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.938081:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.983609:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:0.0:1789760782.983629:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983643:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983653:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983664:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760782.983668:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983676:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983713:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.983720:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b95ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.983808:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507355392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45863us (46871us total) trans 4294972941 rc 0/0 00000100:00100000:0.0:1789760782.983818:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5651 00000100:00100000:0.0:1789760782.983828:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764208807641 time 764253230019 state 0x2 ref 0 00000001:00080000:1.0:1789760782.984084:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00000100:00100000:2.0:1789760782.992363:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760782.992447:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355520 00000100:00100000:0.0:1789760782.992458:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355520 00000100:00100000:0.0:1789760782.992527:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764208807641 time 764253230019 state 0x0 ref 1 00000100:00100000:0.0:1789760782.992544:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764253490435 time 764253230019 state 0x1 ref 1 00000100:00100000:0.0:1789760782.992580:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764253490435 time 764307883752 state 0x0 ref 1 00000100:00100000:0.0:1789760782.992595:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760782.992641:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23027712, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xb007 00010000:00010000:0.0:1789760782.992650:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.992666:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760782.992671:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.992677:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.992712:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.046850:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00002000:00010000:0.0:1789760783.047353:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23027712 -> 23031808 00002000:00010000:0.0:1789760783.047359:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:0.0:1789760783.047362:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:0.0:1789760783.047375:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047388:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047397:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047416:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.047421:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047429:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047448:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.047455:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.047573:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54976us (55215us total) trans 4294972942 rc 0/0 00000100:00100000:0.0:1789760783.047583:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5652 00000100:00100000:0.0:1789760783.047597:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764253490435 time 764307883752 state 0x2 ref 0 00000100:00100000:3.0:1789760783.065833:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.065913:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355648 00000100:00100000:0.0:1789760783.065930:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355648 00000100:00100000:0.0:1789760783.066002:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764253490435 time 764307883752 state 0x0 ref 1 00000100:00100000:0.0:1789760783.066016:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764308144168 time 764307883752 state 0x1 ref 1 00000100:00100000:0.0:1789760783.066049:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764308144168 time 764381352612 state 0x0 ref 1 00000100:00100000:0.0:1789760783.066061:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.066107:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23031808, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.066116:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.066131:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.066135:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.066141:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.066154:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760783.066490:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760783.110812:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:0.0:1789760783.110845:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110859:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110869:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110877:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.110882:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110891:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110929:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.110938:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b95fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.111157:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507355648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45091us (45326us total) trans 4294972943 rc 0/0 00000100:00100000:0.0:1789760783.111170:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5653 00000100:00100000:0.0:1789760783.111181:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764308144168 time 764381352612 state 0x2 ref 0 00000001:00080000:3.0:1789760783.111272:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00000100:00100000:3.0:1789760783.114980:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.115138:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355776 00000100:00100000:0.0:1789760783.115158:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355776 00000100:00100000:0.0:1789760783.115215:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764308144168 time 764381352612 state 0x0 ref 1 00000100:00100000:0.0:1789760783.115230:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764381613028 time 764381352612 state 0x1 ref 1 00000100:00100000:0.0:1789760783.115263:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764381613028 time 764430566455 state 0x0 ref 1 00000100:00100000:0.0:1789760783.115277:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507355776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.115330:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23035904, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.115340:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.115356:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.115361:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.115367:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.115386:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.149321:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00002000:00010000:0.0:1789760783.149393:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:0.0:1789760783.149418:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149432:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149442:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149449:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.149454:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149463:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149494:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.149503:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9603 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.149741:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507355776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34417us (34720us total) trans 4294972944 rc 0/0 00000100:00100000:0.0:1789760783.149757:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5654 00000100:00100000:0.0:1789760783.149766:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764381613028 time 764430566455 state 0x2 ref 0 00000100:00100000:2.0:1789760783.153893:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.154351:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507355904 00000100:00100000:0.0:1789760783.154362:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507355904 00000100:00100000:0.0:1789760783.154416:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764381613028 time 764430566455 state 0x0 ref 1 00000100:00100000:0.0:1789760783.154432:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764430826871 time 764430566455 state 0x1 ref 1 00000100:00100000:0.0:1789760783.154506:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764430826871 time 764469808430 state 0x0 ref 1 00000100:00100000:0.0:1789760783.154519:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.154569:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23040000, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.154578:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.154594:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.154598:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.154604:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.154618:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.197042:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00002000:00010000:0.0:1789760783.197081:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:0.0:1789760783.197099:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197112:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197166:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197174:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.197179:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197485:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197507:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.197514:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b960a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.197609:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507355904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43086us (43726us total) trans 4294972945 rc 0/0 00000100:00100000:0.0:1789760783.197624:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5655 00000100:00100000:0.0:1789760783.197636:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764430826871 time 764469808430 state 0x2 ref 0 00000100:00100000:3.0:1789760783.207003:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.207327:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356032 00000100:00100000:0.0:1789760783.207337:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356032 00000100:00100000:0.0:1789760783.207392:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764430826871 time 764469808430 state 0x0 ref 1 00000100:00100000:0.0:1789760783.207408:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764470068846 time 764469808430 state 0x1 ref 1 00000100:00100000:0.0:1789760783.207500:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764470068846 time 764522802392 state 0x0 ref 1 00000100:00100000:0.0:1789760783.207513:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507356032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.207570:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23044096, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.207579:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.207595:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.207599:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.207606:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.207620:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.239503:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00002000:00010000:0.0:1789760783.239670:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23044096 -> 23048192 00010000:00010000:0.0:1789760783.239686:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239757:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239767:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239774:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.239778:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239788:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239808:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.239815:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9611 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.240006:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507356032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32490us (33007us total) trans 4294972946 rc 0/0 00000100:00100000:0.0:1789760783.240018:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5656 00000100:00100000:0.0:1789760783.240027:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764470068846 time 764522802392 state 0x2 ref 0 00000100:00100000:2.0:1789760783.245048:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.245278:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356160 00000100:00100000:0.0:1789760783.245288:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356160 00000100:00100000:0.0:1789760783.245340:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764470068846 time 764522802392 state 0x0 ref 1 00000100:00100000:0.0:1789760783.245355:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764523062808 time 764522802392 state 0x1 ref 1 00000100:00100000:0.0:1789760783.245417:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764523062808 time 764560719265 state 0x0 ref 1 00000100:00100000:0.0:1789760783.245429:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.245485:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23048192, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.245494:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.245509:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.245523:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.245530:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.245545:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.260296:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760783.276465:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23048192 -> 23052288 00010000:00010000:0.0:1789760783.276494:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276506:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276529:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276536:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.276541:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276550:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276569:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.276577:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9618 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.276669:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507356160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31238us (31627us total) trans 4294972947 rc 0/0 00000100:00100000:0.0:1789760783.276680:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5657 00000100:00100000:0.0:1789760783.276689:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764523062808 time 764560719265 state 0x2 ref 0 00000001:00080000:1.0:1789760783.277973:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00000100:00100000:2.0:1789760783.280822:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.280901:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356288 00000100:00100000:0.0:1789760783.280910:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356288 00000100:00100000:0.0:1789760783.281010:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764523062808 time 764560719265 state 0x0 ref 1 00000100:00100000:0.0:1789760783.281025:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764560979681 time 764560719265 state 0x1 ref 1 00000100:00100000:0.0:1789760783.281080:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764560979681 time 764596383572 state 0x0 ref 1 00000100:00100000:0.0:1789760783.281091:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.281144:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23052288, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.281152:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.281166:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.281170:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.281176:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.281190:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.307456:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00002000:00010000:1.0:1789760783.307567:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:1.0:1789760783.307582:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307595:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307604:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307612:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760783.307617:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307627:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307651:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.307660:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b961f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.307786:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26690us (26968us total) trans 4294972948 rc 0/0 00000100:00100000:1.0:1789760783.307796:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5658 00000100:00100000:1.0:1789760783.307805:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764560979681 time 764596383572 state 0x2 ref 0 00000100:00100000:2.0:1789760783.315644:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.317152:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356416 00000100:00100000:0.0:1789760783.317165:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356416 00000100:00100000:0.0:1789760783.317223:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764560979681 time 764596383572 state 0x0 ref 1 00000100:00100000:0.0:1789760783.317239:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764596643988 time 764596383572 state 0x1 ref 1 00000100:00100000:0.0:1789760783.317314:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764596643988 time 764632615925 state 0x0 ref 1 00000100:00100000:0.0:1789760783.317335:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.317383:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23056384, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.317393:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.317408:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.317413:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.317420:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.317434:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.342438:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:0.0:1789760783.342639:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:0.0:1789760783.342653:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342665:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342674:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342681:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.342685:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342718:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342743:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.342751:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9626 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.343081:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25510us (27200us total) trans 4294972949 rc 0/0 00000100:00100000:0.0:1789760783.343097:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5659 00000100:00100000:0.0:1789760783.343106:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764596643988 time 764632615925 state 0x2 ref 0 00000100:00100000:2.0:1789760783.346337:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.346656:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356544 00000100:00100000:0.0:1789760783.346667:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356544 00000100:00100000:0.0:1789760783.346739:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764596643988 time 764632615925 state 0x0 ref 1 00000100:00100000:0.0:1789760783.346755:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764632876341 time 764632615925 state 0x1 ref 1 00000100:00100000:1.0:1789760783.346971:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764632876341 time 764662271702 state 0x0 ref 1 00000100:00100000:1.0:1789760783.346987:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.347036:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23060480, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:1.0:1789760783.347045:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.347060:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760783.347064:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.347070:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.347084:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.374447:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:0.0:1789760783.374634:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:0.0:1789760783.374649:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374660:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374666:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374676:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.374681:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374690:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374751:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.374759:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b962d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.374866:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27871us (28527us total) trans 4294972950 rc 0/0 00000100:00100000:0.0:1789760783.374876:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5660 00000100:00100000:0.0:1789760783.374885:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764632876341 time 764662271702 state 0x2 ref 0 00000100:00100000:2.0:1789760783.378860:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.378953:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356672 00000100:00100000:1.0:1789760783.378964:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356672 00000100:00100000:1.0:1789760783.379052:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764632876341 time 764662271702 state 0x0 ref 1 00000100:00100000:1.0:1789760783.379066:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764662532118 time 764662271702 state 0x1 ref 1 00000100:00100000:1.0:1789760783.379130:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764662532118 time 764694433006 state 0x0 ref 1 00000100:00100000:1.0:1789760783.379142:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.379187:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23064576, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:1.0:1789760783.379196:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.379211:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760783.379216:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.379222:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.379236:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.407041:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00002000:00010000:1.0:1789760783.407120:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:1.0:1789760783.407136:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407154:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407163:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407170:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760783.407174:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407183:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407203:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.407211:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9634 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.407301:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28156us (28446us total) trans 4294972951 rc 0/0 00000100:00100000:1.0:1789760783.407312:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5661 00000100:00100000:1.0:1789760783.407322:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764662532118 time 764694433006 state 0x2 ref 0 00000100:00100000:2.0:1789760783.410124:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.410181:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356800 00000100:00100000:0.0:1789760783.410191:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356800 00000100:00100000:0.0:1789760783.410242:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764662532118 time 764694433006 state 0x0 ref 1 00000100:00100000:0.0:1789760783.410256:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764694693422 time 764694433006 state 0x1 ref 1 00000100:00100000:0.0:1789760783.410283:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764694693422 time 764725586568 state 0x0 ref 1 00000100:00100000:0.0:1789760783.410297:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.410344:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23068672, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb007 00010000:00010000:0.0:1789760783.410351:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.410366:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.410370:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.410375:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.410389:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.444911:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00002000:00010000:0.0:1789760783.445124:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:0.0:1789760783.445130:0:14618:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:0.0:1789760783.445144:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445158:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445168:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445176:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.445182:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445192:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445212:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445220:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b963b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.445350:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507356800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35048us (35228us total) trans 4294972952 rc 0/0 00000100:00100000:0.0:1789760783.445362:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5662 00000100:00100000:0.0:1789760783.445374:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764694693422 time 764725586568 state 0x2 ref 0 00000100:00100000:2.0:1789760783.448634:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.448737:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507356928 00000100:00100000:0.0:1789760783.448747:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507356928 00000100:00100000:0.0:1789760783.448803:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764694693422 time 764725586568 state 0x0 ref 1 00000100:00100000:0.0:1789760783.448817:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764725846984 time 764725586568 state 0x1 ref 1 00000100:00100000:0.0:1789760783.448854:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764725846984 time 764764157007 state 0x0 ref 1 00000100:00100000:0.0:1789760783.448867:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.448914:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23072768, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.448923:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.448938:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.448943:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.448950:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.448964:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.471971:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00002000:00010000:0.0:1789760783.472081:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23072768 -> 23076864 00010000:00010000:0.0:1789760783.472098:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472111:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472119:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472126:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.472131:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472140:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472166:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.472175:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9642 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.472269:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507356928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23400us (23639us total) trans 4294972953 rc 0/0 00000100:00100000:0.0:1789760783.472278:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5663 00000100:00100000:0.0:1789760783.472288:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764725846984 time 764764157007 state 0x2 ref 0 00000100:00100000:2.0:1789760783.475329:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.475558:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357056 00000100:00100000:1.0:1789760783.475569:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357056 00000100:00100000:1.0:1789760783.475623:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764725846984 time 764764157007 state 0x0 ref 1 00000100:00100000:1.0:1789760783.475638:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764764417423 time 764764157007 state 0x1 ref 1 00000100:00100000:1.0:1789760783.475788:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764764417423 time 764791089506 state 0x0 ref 1 00000100:00100000:1.0:1789760783.475802:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.475850:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23076864, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:1.0:1789760783.475859:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.475874:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760783.475878:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.475885:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.475900:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.497756:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00002000:00010000:0.0:1789760783.497848:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23076864 -> 23080960 00010000:00010000:0.0:1789760783.497864:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497876:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497885:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497892:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.497896:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497911:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497931:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.497938:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9649 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.498064:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22260us (22739us total) trans 4294972954 rc 0/0 00000100:00100000:0.0:1789760783.498073:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5664 00000100:00100000:0.0:1789760783.498081:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764764417423 time 764791089506 state 0x2 ref 0 00000100:00100000:2.0:1789760783.501452:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.501540:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357184 00000100:00100000:0.0:1789760783.501550:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357184 00000100:00100000:0.0:1789760783.501602:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764764417423 time 764791089506 state 0x0 ref 1 00000100:00100000:0.0:1789760783.501621:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764791349922 time 764791089506 state 0x1 ref 1 00000100:00100000:0.0:1789760783.501651:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764791349922 time 764816954744 state 0x0 ref 1 00000100:00100000:0.0:1789760783.501664:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507357184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.501734:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23080960, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.501743:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501759:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.501764:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.501770:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501785:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.531804:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00002000:00010000:0.0:1789760783.531916:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23080960 -> 23085056 00010000:00010000:0.0:1789760783.531934:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.531947:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.531955:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.531962:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.531968:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.531977:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.531998:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.532007:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9650 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.532140:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507357184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30473us (30693us total) trans 4294972955 rc 0/0 00000100:00100000:0.0:1789760783.532150:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5665 00000100:00100000:0.0:1789760783.532159:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764791349922 time 764816954744 state 0x2 ref 0 00000100:00100000:2.0:1789760783.538417:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.538511:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357312 00000100:00100000:0.0:1789760783.538526:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357312 00000100:00100000:0.0:1789760783.538582:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764791349922 time 764816954744 state 0x0 ref 1 00000100:00100000:0.0:1789760783.538598:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764817215160 time 764816954744 state 0x1 ref 1 00000100:00100000:0.0:1789760783.538625:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764817215160 time 764853928882 state 0x0 ref 1 00000100:00100000:0.0:1789760783.538642:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.538689:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23085056, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.538778:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.538794:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.538799:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.538805:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.538825:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.563679:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00002000:00010000:0.0:1789760783.563778:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:0.0:1789760783.563792:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563805:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563814:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563821:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.563826:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563835:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563855:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.563863:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9657 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.563952:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25306us (25540us total) trans 4294972956 rc 0/0 00000100:00100000:0.0:1789760783.563963:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5666 00000100:00100000:0.0:1789760783.563972:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764817215160 time 764853928882 state 0x2 ref 0 00000100:00100000:3.0:1789760783.571231:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.571335:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357440 00000100:00100000:0.0:1789760783.571346:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357440 00000100:00100000:0.0:1789760783.571400:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764817215160 time 764853928882 state 0x0 ref 1 00000100:00100000:0.0:1789760783.571414:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764854189298 time 764853928882 state 0x1 ref 1 00000100:00100000:0.0:1789760783.571446:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764854189298 time 764886749120 state 0x0 ref 1 00000100:00100000:0.0:1789760783.571458:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507357440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.571514:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23089152, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.571523:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.571539:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.571544:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.571553:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.571568:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.614767:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00002000:00010000:0.0:1789760783.614891:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:0.0:1789760783.615058:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615071:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615081:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615089:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.615094:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615102:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615122:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.615130:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b965e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.615292:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507357440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43830us (44068us total) trans 4294972957 rc 0/0 00000100:00100000:0.0:1789760783.615307:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5667 00000100:00100000:0.0:1789760783.615317:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764854189298 time 764886749120 state 0x2 ref 0 00000100:00100000:0.0:1789760783.621015:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.621149:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357568 00000100:00100000:1.0:1789760783.621159:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357568 00000100:00100000:1.0:1789760783.621215:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764854189298 time 764886749120 state 0x0 ref 1 00000100:00100000:1.0:1789760783.621231:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764887009536 time 764886749120 state 0x1 ref 1 00000100:00100000:1.0:1789760783.621269:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764887009536 time 764936571606 state 0x0 ref 1 00000100:00100000:1.0:1789760783.621282:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507357568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.621332:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23093248, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:1.0:1789760783.621341:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.621357:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760783.621361:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.621367:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.621380:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.667250:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:1.0:1789760783.667327:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:1.0:1789760783.667341:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667354:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667363:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667369:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760783.667374:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667384:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667403:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.667410:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9665 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.667619:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507357568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46332us (46606us total) trans 4294972958 rc 0/0 00000100:00100000:1.0:1789760783.667632:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5668 00000100:00100000:1.0:1789760783.667642:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764887009536 time 764936571606 state 0x2 ref 0 00000100:00100000:2.0:1789760783.670881:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.671301:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357696 00000100:00100000:0.0:1789760783.671312:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357696 00000100:00100000:0.0:1789760783.671368:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764887009536 time 764936571606 state 0x0 ref 1 00000100:00100000:0.0:1789760783.671383:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764936832022 time 764936571606 state 0x1 ref 1 00000100:00100000:0.0:1789760783.671405:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764936832022 time 764986708930 state 0x0 ref 1 00000100:00100000:0.0:1789760783.671417:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507357696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.671464:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23097344, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.671563:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671581:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760783.671585:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.671592:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671606:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.718492:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00002000:00010000:0.0:1789760783.718736:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23097344 -> 23101440 00010000:00010000:0.0:1789760783.718752:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718765:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718785:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718793:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.718798:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718807:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718829:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.718837:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b966c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.718930:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507357696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47510us (48057us total) trans 4294972959 rc 0/0 00000100:00100000:0.0:1789760783.718940:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5669 00000100:00100000:0.0:1789760783.718949:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764936832022 time 764986708930 state 0x2 ref 0 00000100:00100000:3.0:1789760783.724010:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.724154:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357824 00000100:00100000:0.0:1789760783.724165:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357824 00000100:00100000:0.0:1789760783.724218:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764936832022 time 764986708930 state 0x0 ref 1 00000100:00100000:0.0:1789760783.724233:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764986969346 time 764986708930 state 0x1 ref 1 00000100:00100000:0.0:1789760783.724261:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764986969346 time 765039564412 state 0x0 ref 1 00000100:00100000:0.0:1789760783.724274:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.724322:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23101440, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.724331:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.724347:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760783.724351:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.724357:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.724372:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.773049:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00002000:00010000:1.0:1789760783.773205:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:1.0:1789760783.773220:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773234:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773243:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773250:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760783.773256:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773265:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773285:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.773292:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9673 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.773678:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507357824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49401us (49672us total) trans 4294972960 rc 0/0 00000100:00100000:1.0:1789760783.773689:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5670 00000100:00100000:1.0:1789760783.773736:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764986969346 time 765039564412 state 0x2 ref 0 00000100:00100000:0.0:1789760783.783042:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.783135:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507357952 00000100:00100000:1.0:1789760783.783144:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507357952 00000100:00100000:1.0:1789760783.783194:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 764986969346 time 765039564412 state 0x0 ref 1 00000100:00100000:1.0:1789760783.783207:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765039824828 time 765039564412 state 0x1 ref 1 00000100:00100000:1.0:1789760783.783231:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765039824828 time 765098534911 state 0x0 ref 1 00000100:00100000:1.0:1789760783.783243:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507357952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.783289:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23105536, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:1.0:1789760783.783298:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.783313:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760783.783318:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.783324:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.783337:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.817645:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:2.0:1789760783.817668:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817681:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817689:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817733:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760783.817738:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817747:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817780:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.817789:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b967a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.817881:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507357952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34633us (34842us total) trans 4294972961 rc 0/0 00000100:00100000:2.0:1789760783.817891:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5671 00000100:00100000:2.0:1789760783.817901:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765039824828 time 765098534911 state 0x2 ref 0 00000001:00080000:0.0:1789760783.818037:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00000100:00100000:2.0:1789760783.823132:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.823229:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358080 00000100:00100000:0.0:1789760783.823238:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358080 00000100:00100000:0.0:1789760783.823292:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765039824828 time 765098534911 state 0x0 ref 1 00000100:00100000:0.0:1789760783.823311:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765098795327 time 765098534911 state 0x1 ref 1 00000100:00100000:0.0:1789760783.823377:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765098795327 time 765138679579 state 0x0 ref 1 00000100:00100000:0.0:1789760783.823390:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.823439:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23109632, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.823449:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.823465:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760783.823479:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.823486:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.823501:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.860304:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00002000:00010000:0.0:1789760783.860387:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:0.0:1789760783.860402:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860416:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860425:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860438:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.860442:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860452:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860480:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.860489:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9681 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.860680:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37286us (37551us total) trans 4294972962 rc 0/0 00000100:00100000:0.0:1789760783.860726:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5672 00000100:00100000:0.0:1789760783.860736:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765098795327 time 765138679579 state 0x2 ref 0 00000100:00100000:2.0:1789760783.864283:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.864349:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358208 00000100:00100000:1.0:1789760783.864358:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358208 00000100:00100000:1.0:1789760783.864408:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765098795327 time 765138679579 state 0x0 ref 1 00000100:00100000:1.0:1789760783.864424:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765138939995 time 765138679579 state 0x1 ref 1 00000100:00100000:1.0:1789760783.864455:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765138939995 time 765179758667 state 0x0 ref 1 00000100:00100000:1.0:1789760783.864467:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.864620:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23113728, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:1.0:1789760783.864630:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.864646:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760783.864650:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.864657:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.864673:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.908854:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:1.0:1789760783.909006:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:1.0:1789760783.909022:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909035:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909044:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909052:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760783.909057:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909067:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909086:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.909094:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9688 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.909240:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44768us (44959us total) trans 4294972963 rc 0/0 00000100:00100000:1.0:1789760783.909250:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5673 00000100:00100000:1.0:1789760783.909259:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765138939995 time 765179758667 state 0x2 ref 0 00000100:00100000:3.0:1789760783.914626:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760783.914765:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358336 00000100:00100000:2.0:1789760783.914775:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358336 00000100:00100000:2.0:1789760783.914829:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765138939995 time 765179758667 state 0x0 ref 1 00000100:00100000:2.0:1789760783.914848:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765180019083 time 765179758667 state 0x1 ref 1 00000100:00100000:2.0:1789760783.914875:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765180019083 time 765230178127 state 0x0 ref 1 00000100:00100000:2.0:1789760783.914890:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760783.914939:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23117824, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:2.0:1789760783.914947:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.914962:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760783.914967:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.914972:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.914995:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.951148:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00002000:00010000:2.0:1789760783.951327:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23117824 -> 23121920 00010000:00010000:2.0:1789760783.951342:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951353:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951361:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951368:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760783.951374:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951383:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951402:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.951410:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b968f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.952060:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36622us (36892us total) trans 4294972964 rc 0/0 00000100:00100000:2.0:1789760783.952071:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5674 00000100:00100000:2.0:1789760783.952081:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765180019083 time 765230178127 state 0x2 ref 0 00000100:00100000:3.0:1789760783.958999:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760783.959204:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358464 00000100:00100000:0.0:1789760783.959214:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358464 00000100:00100000:0.0:1789760783.959269:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765180019083 time 765230178127 state 0x0 ref 1 00000100:00100000:0.0:1789760783.959284:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765230438543 time 765230178127 state 0x1 ref 1 00000100:00100000:0.0:1789760783.959318:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765230438543 time 765274621277 state 0x0 ref 1 00000100:00100000:0.0:1789760783.959332:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760783.959379:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23121920, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:0.0:1789760783.959388:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.959404:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760783.959409:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.959416:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.959431:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.985871:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00002000:00010000:0.0:1789760783.986094:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:0.0:1789760783.986111:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986123:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986132:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986138:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760783.986143:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986151:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986171:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986179:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9696 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.986271:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26937us (27276us total) trans 4294972965 rc 0/0 00000100:00100000:0.0:1789760783.986280:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5675 00000100:00100000:0.0:1789760783.986288:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765230438543 time 765274621277 state 0x2 ref 0 00000100:00100000:0.0:1789760783.992359:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760783.992461:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358592 00000100:00100000:1.0:1789760783.992481:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358592 00000100:00100000:1.0:1789760783.992599:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765230438543 time 765274621277 state 0x0 ref 1 00000100:00100000:1.0:1789760783.992614:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765274881693 time 765274621277 state 0x1 ref 1 00000100:00100000:1.0:1789760783.992684:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765274881693 time 765307986206 state 0x0 ref 1 00000100:00100000:1.0:1789760783.992718:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760783.992785:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23126016, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xb807 00010000:00010000:1.0:1789760783.992794:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.992812:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760783.992816:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.992822:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.992837:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.028725:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00002000:00010000:0.0:1789760784.028909:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23126016 -> 23130112 00002000:00010000:0.0:1789760784.028979:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:0.0:1789760784.028983:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:0.0:1789760784.028999:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029012:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029021:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029029:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.029035:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029043:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029064:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.029072:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b969d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.029203:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36503us (36850us total) trans 4294972966 rc 0/0 00000100:00100000:0.0:1789760784.029214:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5676 00000100:00100000:0.0:1789760784.029223:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765274881693 time 765307986206 state 0x2 ref 0 00000100:00100000:3.0:1789760784.033001:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.033077:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358720 00000100:00100000:2.0:1789760784.033086:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358720 00000100:00100000:2.0:1789760784.033143:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765274881693 time 765307986206 state 0x0 ref 1 00000100:00100000:2.0:1789760784.033158:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765308246622 time 765307986206 state 0x1 ref 1 00000100:00100000:2.0:1789760784.033192:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765308246622 time 765348494926 state 0x0 ref 1 00000100:00100000:2.0:1789760784.033205:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760784.033252:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23130112, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:2.0:1789760784.033262:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.033278:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760784.033282:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.033289:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.033304:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.066312:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00002000:00010000:2.0:1789760784.066421:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:2.0:1789760784.066436:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066450:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066459:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066467:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760784.066482:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066491:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066582:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.066592:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b96a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.066761:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33553us (33764us total) trans 4294972967 rc 0/0 00000100:00100000:2.0:1789760784.066896:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5677 00000100:00100000:2.0:1789760784.066907:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765308246622 time 765348494926 state 0x2 ref 0 00000100:00100000:1.0:1789760784.071262:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.071412:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358848 00000100:00100000:2.0:1789760784.071422:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358848 00000100:00100000:2.0:1789760784.071484:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765308246622 time 765348494926 state 0x0 ref 1 00000100:00100000:2.0:1789760784.071499:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765348755342 time 765348494926 state 0x1 ref 1 00000100:00100000:2.0:1789760784.071535:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765348755342 time 765386838715 state 0x0 ref 1 00000100:00100000:2.0:1789760784.071547:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507358848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760784.071589:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23134208, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:2.0:1789760784.071598:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071613:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760784.071617:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.071624:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071638:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.089302:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.089342:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.089373:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.089397:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.089418:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.089446:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359104 00000100:00100000:2.0:1789760784.089455:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359104 00000100:00100000:2.0:1789760784.089532:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359360 00000100:00100000:2.0:1789760784.089536:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359360 00000100:00100000:2.0:1789760784.089556:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359232 00000100:00100000:2.0:1789760784.089559:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359232 00000100:00100000:2.0:1789760784.089571:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359488 00000100:00100000:2.0:1789760784.089573:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359488 00000100:00100000:2.0:1789760784.089591:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 968 00000100:00100000:2.0:1789760784.089601:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507359104:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760784.089675:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+8:2419:x1876699507359104:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 75us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.089683:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 968 00000100:00100000:2.0:1789760784.089717:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 969 00000100:00100000:2.0:1789760784.089724:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507359360:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760784.089751:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507359360:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 28us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.089757:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 969 00000100:00100000:2.0:1789760784.089781:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 970 00000100:00100000:2.0:1789760784.089788:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227869c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507359232:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760784.089814:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227869c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507359232:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 27us (416us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.089820:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 970 00000100:00100000:2.0:1789760784.089832:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 971 00000100:00100000:2.0:1789760784.089839:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507359488:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760784.089862:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507359488:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 24us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.089867:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 971 00000100:00100000:2.0:1789760784.089886:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507358976 00000100:00100000:2.0:1789760784.089890:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507358976 00000100:00100000:2.0:1789760784.089928:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 484 00000100:00100000:2.0:1789760784.089936:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507358976:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760784.089961:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507358976:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 26us (588us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.089967:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 484 00000100:00080000:2.0:1789760784.089999:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760784.090002:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1789760784.111424:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:2.0:1789760784.111444:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111457:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111466:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111498:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760784.111511:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111522:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111544:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.111552:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b96ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.111848:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507358848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40294us (40587us total) trans 4294972968 rc 0/0 00000100:00100000:2.0:1789760784.111859:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5678 00000100:00100000:2.0:1789760784.111869:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765348755342 time 765386838715 state 0x2 ref 0 00000001:00080000:3.0:1789760784.112062:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00000100:00100000:2.0:1789760784.115449:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.115642:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359616 00000100:00100000:0.0:1789760784.115652:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359616 00000100:00100000:0.0:1789760784.116017:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765348755342 time 765386838715 state 0x0 ref 1 00000100:00100000:0.0:1789760784.116034:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765387099131 time 765386838715 state 0x1 ref 1 00000100:00100000:0.0:1789760784.116055:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765387099131 time 765431358977 state 0x0 ref 1 00000100:00100000:0.0:1789760784.116073:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507359616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.116121:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23138304, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.116130:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.116145:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.116150:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.116156:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.116170:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.148868:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:3.0:1789760784.148887:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.148902:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.148912:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.148982:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.148988:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.148998:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.149018:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.149026:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b96b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.149121:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507359616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33047us (33678us total) trans 4294972969 rc 0/0 00000100:00100000:3.0:1789760784.149131:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5679 00000100:00100000:3.0:1789760784.149140:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765387099131 time 765431358977 state 0x2 ref 0 00000001:00080000:3.0:1789760784.149177:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00000100:00100000:1.0:1789760784.152960:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.153050:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359744 00000100:00100000:2.0:1789760784.153061:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359744 00000100:00100000:2.0:1789760784.153113:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765387099131 time 765431358977 state 0x0 ref 1 00000100:00100000:2.0:1789760784.153128:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765431619393 time 765431358977 state 0x1 ref 1 00000100:00100000:2.0:1789760784.153157:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765431619393 time 765468460559 state 0x0 ref 1 00000100:00100000:2.0:1789760784.153169:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507359744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760784.153217:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23142400, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:2.0:1789760784.153227:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.153243:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760784.153248:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.153254:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.153269:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.184167:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00002000:00010000:2.0:1789760784.184265:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:2.0:1789760784.184281:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184294:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184303:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184310:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760784.184315:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184324:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184348:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.184356:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.184450:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507359744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31278us (31494us total) trans 4294972970 rc 0/0 00000100:00100000:2.0:1789760784.184460:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5680 00000100:00100000:2.0:1789760784.184479:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765431619393 time 765468460559 state 0x2 ref 0 00000100:00100000:2.0:1789760784.190103:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.190619:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507359872 00000100:00100000:0.0:1789760784.190630:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507359872 00000100:00100000:0.0:1789760784.190683:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765431619393 time 765468460559 state 0x0 ref 1 00000100:00100000:0.0:1789760784.190732:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765468720975 time 765468460559 state 0x1 ref 1 00000100:00100000:0.0:1789760784.190755:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765468720975 time 765506058474 state 0x0 ref 1 00000100:00100000:0.0:1789760784.190768:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507359872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.190815:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23146496, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.190825:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.190847:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.190852:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.190858:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.190871:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.225460:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00002000:00010000:0.0:1789760784.225612:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:0.0:1789760784.225633:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225648:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225657:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225664:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.225668:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225677:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225738:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.225748:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.225997:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507359872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35226us (35897us total) trans 4294972971 rc 0/0 00000100:00100000:0.0:1789760784.226008:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5681 00000100:00100000:0.0:1789760784.226017:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765468720975 time 765506058474 state 0x2 ref 0 00000100:00100000:3.0:1789760784.236746:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.236921:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360000 00000100:00100000:0.0:1789760784.236944:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360000 00000100:00100000:0.0:1789760784.237001:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765468720975 time 765506058474 state 0x0 ref 1 00000100:00100000:0.0:1789760784.237017:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765506318890 time 765506058474 state 0x1 ref 1 00000100:00100000:0.0:1789760784.237047:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765506318890 time 765552350358 state 0x0 ref 1 00000100:00100000:0.0:1789760784.237061:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507360000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.237117:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23150592, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.237138:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.237152:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.237157:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.237164:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.237184:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.264349:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00002000:00010000:3.0:1789760784.264450:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:3.0:1789760784.264465:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264488:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264497:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264513:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.264518:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264528:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264546:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.264561:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b96c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.264648:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507360000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27585us (27907us total) trans 4294972972 rc 0/0 00000100:00100000:3.0:1789760784.264658:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5682 00000100:00100000:3.0:1789760784.264667:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765506318890 time 765552350358 state 0x2 ref 0 00000100:00100000:1.0:1789760784.270477:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.270667:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360128 00000100:00100000:2.0:1789760784.270677:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360128 00000100:00100000:2.0:1789760784.270765:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765506318890 time 765552350358 state 0x0 ref 1 00000100:00100000:2.0:1789760784.270781:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765552610774 time 765552350358 state 0x1 ref 1 00000100:00100000:2.0:1789760784.270844:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765552610774 time 765586146804 state 0x0 ref 1 00000100:00100000:2.0:1789760784.270859:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507360128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760784.270906:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23154688, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:2.0:1789760784.270915:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.270937:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760784.270942:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.270949:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.270963:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.284162:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760784.302576:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00002000:00010000:3.0:1789760784.302741:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:3.0:1789760784.302756:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302769:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302785:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302793:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.302799:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302809:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302829:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.302837:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.302955:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507360128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32095us (32493us total) trans 4294972973 rc 0/0 00000100:00100000:3.0:1789760784.302967:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5683 00000100:00100000:3.0:1789760784.302977:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765552610774 time 765586146804 state 0x2 ref 0 00000100:00100000:1.0:1789760784.308772:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.308858:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360256 00000100:00100000:0.0:1789760784.308868:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360256 00000100:00100000:0.0:1789760784.308923:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765552610774 time 765586146804 state 0x0 ref 1 00000100:00100000:0.0:1789760784.308938:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765586407220 time 765586146804 state 0x1 ref 1 00000100:00100000:0.0:1789760784.308998:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765586407220 time 765624301536 state 0x0 ref 1 00000100:00100000:0.0:1789760784.309008:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.309047:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23158784, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.309053:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.309065:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.309068:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.309073:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.309087:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.341645:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00002000:00010000:2.0:1789760784.342148:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23158784 -> 23162880 00010000:00010000:2.0:1789760784.342162:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342176:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342184:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342191:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760784.342195:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342204:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342228:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342236:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b96d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.342340:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33327us (33571us total) trans 4294972974 rc 0/0 00000100:00100000:2.0:1789760784.342349:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5684 00000100:00100000:2.0:1789760784.342359:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765586407220 time 765624301536 state 0x2 ref 0 00000100:00100000:0.0:1789760784.348754:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.348941:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360384 00000100:00100000:3.0:1789760784.348951:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360384 00000100:00100000:3.0:1789760784.349010:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765586407220 time 765624301536 state 0x0 ref 1 00000100:00100000:3.0:1789760784.349024:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765624561952 time 765624301536 state 0x1 ref 1 00000100:00100000:3.0:1789760784.349051:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765624561952 time 765664354328 state 0x0 ref 1 00000100:00100000:3.0:1789760784.349063:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.349108:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23162880, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.349117:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.349131:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760784.349135:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.349141:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.349156:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.416248:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00002000:00010000:3.0:1789760784.416395:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23162880 -> 23166976 00010000:00010000:3.0:1789760784.416411:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416421:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416429:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416435:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.416441:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416448:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416477:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416484:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.416680:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 67614us (67929us total) trans 4294972975 rc 0/0 00000100:00100000:3.0:1789760784.416738:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5685 00000100:00100000:3.0:1789760784.416748:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765624561952 time 765664354328 state 0x2 ref 0 00000100:00100000:1.0:1789760784.419911:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.419964:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360512 00000100:00100000:3.0:1789760784.419973:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360512 00000100:00100000:3.0:1789760784.421087:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765624561952 time 765664354328 state 0x0 ref 1 00000100:00100000:3.0:1789760784.421104:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765664614744 time 765664354328 state 0x1 ref 1 00000100:00100000:3.0:1789760784.421141:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765664614744 time 765736445069 state 0x0 ref 1 00000100:00100000:3.0:1789760784.421154:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507360512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.421200:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23166976, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.421210:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.421225:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760784.421229:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.421236:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.421248:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.469447:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:0.0:1789760784.469764:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23166976 -> 23171072 00010000:00010000:0.0:1789760784.469781:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469794:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469803:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469810:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.469814:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469823:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469845:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.469852:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b96e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.469958:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507360512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48802us (50263us total) trans 4294972976 rc 0/0 00000100:00100000:0.0:1789760784.469969:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5686 00000100:00100000:0.0:1789760784.469978:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765664614744 time 765736445069 state 0x2 ref 0 00000100:00100000:1.0:1789760784.476234:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.476557:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360640 00000100:00100000:2.0:1789760784.476567:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360640 00000100:00100000:2.0:1789760784.476625:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765664614744 time 765736445069 state 0x0 ref 1 00000100:00100000:2.0:1789760784.476641:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765736705485 time 765736445069 state 0x1 ref 1 00000100:00100000:2.0:1789760784.476672:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765736705485 time 765791975736 state 0x0 ref 1 00000100:00100000:2.0:1789760784.476686:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760784.476773:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23171072, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:2.0:1789760784.476783:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.476806:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760784.476811:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.476817:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.476832:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.508510:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23171072 -> 23175168 00010000:00010000:0.0:1789760784.508533:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508545:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508555:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508562:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.508567:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508577:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508598:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.508606:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b96ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.508741:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507360640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32052us (32512us total) trans 4294972977 rc 0/0 00000100:00100000:0.0:1789760784.508753:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5687 00000100:00100000:0.0:1789760784.508764:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765736705485 time 765791975736 state 0x2 ref 0 00000001:00080000:3.0:1789760784.513111:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00000100:00100000:0.0:1789760784.513336:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.513738:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360768 00000100:00100000:3.0:1789760784.513748:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360768 00000100:00100000:3.0:1789760784.513802:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765736705485 time 765791975736 state 0x0 ref 1 00000100:00100000:3.0:1789760784.513827:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765792236152 time 765791975736 state 0x1 ref 1 00000100:00100000:3.0:1789760784.513855:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765792236152 time 765829158610 state 0x0 ref 1 00000100:00100000:3.0:1789760784.513869:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507360768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.513927:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23175168, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.513936:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.513951:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760784.513956:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.513963:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.513976:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.540200:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23175168 -> 23179264 00010000:00010000:0.0:1789760784.540222:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540235:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540243:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540251:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.540255:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540264:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540296:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.540309:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b96f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.540408:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507360768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26536us (27071us total) trans 4294972978 rc 0/0 00000100:00100000:0.0:1789760784.540418:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5688 00000100:00100000:0.0:1789760784.540428:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765792236152 time 765829158610 state 0x2 ref 0 00000001:00080000:2.0:1789760784.540621:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00000100:00100000:1.0:1789760784.545003:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.545119:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507360896 00000100:00100000:0.0:1789760784.545130:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507360896 00000100:00100000:0.0:1789760784.545186:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765792236152 time 765829158610 state 0x0 ref 1 00000100:00100000:0.0:1789760784.545201:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765829419026 time 765829158610 state 0x1 ref 1 00000100:00100000:0.0:1789760784.545231:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765829419026 time 765860534541 state 0x0 ref 1 00000100:00100000:0.0:1789760784.545243:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.545291:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23179264, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.545301:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.545316:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.545321:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.545326:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.545343:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.576201:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00002000:00010000:1.0:1789760784.576279:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:1.0:1789760784.576293:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576306:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576316:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576323:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760784.576327:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576335:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576351:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.576359:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b96f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.576486:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507360896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31240us (31489us total) trans 4294972979 rc 0/0 00000100:00100000:1.0:1789760784.576498:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5689 00000100:00100000:1.0:1789760784.576515:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765829419026 time 765860534541 state 0x2 ref 0 00000100:00100000:1.0:1789760784.581264:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.581346:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361024 00000100:00100000:0.0:1789760784.581356:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361024 00000100:00100000:0.0:1789760784.581404:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765829419026 time 765860534541 state 0x0 ref 1 00000100:00100000:0.0:1789760784.581418:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765860794957 time 765860534541 state 0x1 ref 1 00000100:00100000:0.0:1789760784.581488:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765860794957 time 765896790889 state 0x0 ref 1 00000100:00100000:0.0:1789760784.581501:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507361024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.581561:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23183360, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.581571:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.581587:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760784.581592:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.581599:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.581613:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.604122:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538292608:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604195:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604242:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538293120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604271:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604290:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108020380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538293632:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604313:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604331:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108020700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538294144:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604351:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604374:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1789760784.604380:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538292992:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604403:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604421:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108020a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538293504:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604441:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604459:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108023b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538294016:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604495:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604539:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538292736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604562:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604579:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538293248:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604599:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604614:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538293760:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604633:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604649:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1789760784.604654:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538292864:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.604673:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.604689:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538293376:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607156:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.607194:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538293888:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607218:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.607250:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538294272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760784.607280:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.607301:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760784.607308:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538294400:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760784.607330:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.607371:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292608 00000100:00100000:1.0:1789760784.607380:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292608 00000100:00100000:1.0:1789760784.607423:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1789760784.607432:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538292608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607499:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3462:x1876699538292608:12345-0@lo:400:kworker.0 Request processed in 67us (3304us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607516:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1789760784.607552:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293120 00000100:00100000:1.0:1789760784.607557:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293120 00000100:00100000:1.0:1789760784.607580:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292864 00000100:00100000:1.0:1789760784.607584:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292864 00000100:00100000:1.0:1789760784.607600:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789760784.607606:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538293120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607641:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538293120:12345-0@lo:400:kworker.0 Request processed in 35us (3369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607648:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789760784.607661:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789760784.607667:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108105f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538292864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607717:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108105f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538292864:12345-0@lo:400:kworker.0 Request processed in 50us (3043us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607724:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789760784.607741:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293632 00000100:00100000:1.0:1789760784.607745:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293632 00000100:00100000:1.0:1789760784.607777:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294144 00000100:00100000:1.0:1789760784.607780:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294144 00000100:00100000:1.0:1789760784.607793:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293504 00000100:00100000:1.0:1789760784.607796:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293504 00000100:00100000:1.0:1789760784.607809:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1789760784.607816:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538293632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607849:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538293632:12345-0@lo:400:kworker.0 Request processed in 33us (3534us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607855:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1789760784.607866:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294016 00000100:00100000:1.0:1789760784.607869:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294016 00000100:00100000:1.0:1789760784.607883:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789760784.607888:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538294144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607915:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538294144:12345-0@lo:400:kworker.0 Request processed in 27us (3564us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607921:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789760784.607932:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1789760784.607937:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538293504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.607963:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538293504:12345-0@lo:400:kworker.0 Request processed in 26us (3521us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.607969:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1789760784.607984:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1789760784.607990:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538294016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608018:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227829880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538294016:12345-0@lo:400:kworker.0 Request processed in 28us (3522us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608023:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1789760784.608046:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292992 00000100:00100000:1.0:1789760784.608050:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292992 00000100:00100000:1.0:1789760784.608068:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538292736 00000100:00100000:1.0:1789760784.608071:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538292736 00000100:00100000:1.0:1789760784.608083:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293248 00000100:00100000:1.0:1789760784.608085:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293248 00000100:00100000:1.0:1789760784.608096:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293760 00000100:00100000:1.0:1789760784.608098:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293760 00000100:00100000:1.0:1789760784.608109:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293376 00000100:00100000:1.0:1789760784.608112:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293376 00000100:00100000:1.0:1789760784.608124:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538293888 00000100:00100000:1.0:1789760784.608126:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538293888 00000100:00100000:1.0:1789760784.608141:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 972 00000100:00100000:1.0:1789760784.608148:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538292992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608180:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227828e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3462:x1876699538292992:12345-0@lo:400:kworker.0 Request processed in 32us (3776us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608185:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 972 00000100:00100000:1.0:1789760784.608197:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 973 00000100:00100000:1.0:1789760784.608203:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538292736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608228:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22782b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3463:x1876699538292736:12345-0@lo:400:kworker.0 Request processed in 26us (3665us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608233:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 973 00000100:00100000:1.0:1789760784.608246:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 974 00000100:00100000:1.0:1789760784.608253:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538293248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608281:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3463:x1876699538293248:12345-0@lo:400:kworker.0 Request processed in 29us (3681us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608286:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 974 00000100:00100000:1.0:1789760784.608297:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 975 00000100:00100000:1.0:1789760784.608304:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108104000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538293760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608330:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108104000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3463:x1876699538293760:12345-0@lo:400:kworker.0 Request processed in 27us (3696us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608336:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 975 00000100:00100000:1.0:1789760784.608347:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 976 00000100:00100000:1.0:1789760784.608353:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538293376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608379:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538293376:12345-0@lo:400:kworker.0 Request processed in 27us (1226us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608384:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 976 00000100:00100000:1.0:1789760784.608395:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 977 00000100:00100000:1.0:1789760784.608401:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538293888:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.608426:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108106680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538293888:12345-0@lo:400:kworker.0 Request processed in 25us (1207us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.608432:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 977 00000100:00100000:1.0:1789760784.608986:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294272 00000100:00100000:1.0:1789760784.608992:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294272 00000100:00100000:1.0:1789760784.609024:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294400 00000100:00100000:1.0:1789760784.609027:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294400 00000100:00100000:1.0:1789760784.609041:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789760784.609048:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538294272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760784.609075:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760784.609113:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234628e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538294272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 65us (1832us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.609120:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789760784.609132:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789760784.609139:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538294400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760784.609148:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760784.609172:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538294400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 35us (1842us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.609178:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00080000:1.0:1789760784.609205:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760784.609209:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760784.609212:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760784.609214:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760784.609356:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3462:1876699538292992:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609370:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108020a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538293504:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609384:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108023b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538294016:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609394:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a12369fc-e31f-4eec-83dc-16c04178fd0b:3462:1876699538292608:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609405:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538293120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609416:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108020380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538293632:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609426:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108020700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538294144:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609486:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538294400:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760784.609499:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538294272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760784.609513:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538292864:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609523:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538293376:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609533:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538293888:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609543:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3463:1876699538292736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609553:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3463:1876699538293248:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.609562:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3463:1876699538293760:0@lo:400:kworker.0 00002000:00010000:0.0:1789760784.619866:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:0.0:1789760784.619888:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619901:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619911:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619918:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.619924:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619933:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619952:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619960:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b96ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.620054:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507361024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38550us (38876us total) trans 4294972980 rc 0/0 00000100:00100000:0.0:1789760784.620064:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5690 00000100:00100000:0.0:1789760784.620074:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765860794957 time 765896790889 state 0x2 ref 0 00000100:00100000:0.0:1789760784.622903:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.623016:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361152 00000100:00100000:3.0:1789760784.623025:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361152 00000100:00100000:3.0:1789760784.623073:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765860794957 time 765896790889 state 0x0 ref 1 00000100:00100000:3.0:1789760784.623088:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765897051305 time 765896790889 state 0x1 ref 1 00000100:00100000:3.0:1789760784.623118:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765897051305 time 765938421113 state 0x0 ref 1 00000100:00100000:3.0:1789760784.623128:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507361152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.623173:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23187456, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.623182:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.623198:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760784.623202:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.623208:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.623221:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.624135:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00000001:00080000:2.0:1789760784.652458:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00002000:00010000:3.0:1789760784.652630:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:3.0:1789760784.652645:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652659:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652668:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652675:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.652680:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652689:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652728:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652737:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9706 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.652867:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507361152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29734us (29967us total) trans 4294972981 rc 0/0 00000100:00100000:3.0:1789760784.652883:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5691 00000100:00100000:3.0:1789760784.652893:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765897051305 time 765938421113 state 0x2 ref 0 00000100:00100000:3.0:1789760784.656019:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760784.656491:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361280 00000100:00100000:1.0:1789760784.656510:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361280 00000100:00100000:1.0:1789760784.656570:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765897051305 time 765938421113 state 0x0 ref 1 00000100:00100000:1.0:1789760784.656586:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765938681529 time 765938421113 state 0x1 ref 1 00000100:00100000:1.0:1789760784.656618:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765938681529 time 765971921571 state 0x0 ref 1 00000100:00100000:1.0:1789760784.656631:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507361280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760784.656678:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23191552, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:1.0:1789760784.656686:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.657008:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760784.657013:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.657020:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.657034:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.687247:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00002000:00010000:1.0:1789760784.687409:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:1.0:1789760784.687421:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687431:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687437:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687442:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760784.687445:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687452:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687479:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.687486:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b970d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.687672:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507361280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31037us (31657us total) trans 4294972982 rc 0/0 00000100:00100000:1.0:1789760784.687683:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5692 00000100:00100000:1.0:1789760784.687728:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765938681529 time 765971921571 state 0x2 ref 0 00000100:00100000:1.0:1789760784.691091:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.691407:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361408 00000100:00100000:0.0:1789760784.691416:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361408 00000100:00100000:0.0:1789760784.691460:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765938681529 time 765971921571 state 0x0 ref 1 00000100:00100000:0.0:1789760784.691486:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765972181987 time 765971921571 state 0x1 ref 1 00000100:00100000:0.0:1789760784.691518:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765972181987 time 766006821103 state 0x0 ref 1 00000100:00100000:0.0:1789760784.691530:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507361408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.691577:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23195648, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.691585:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.691601:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760784.691606:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.691612:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.691625:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.722151:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00002000:00010000:3.0:1789760784.722268:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:3.0:1789760784.722284:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722298:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722308:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722315:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760784.722320:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722329:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722348:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.722355:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9714 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.722445:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507361408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30911us (31359us total) trans 4294972983 rc 0/0 00000100:00100000:3.0:1789760784.722456:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5693 00000100:00100000:3.0:1789760784.722476:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765972181987 time 766006821103 state 0x2 ref 0 00000100:00100000:0.0:1789760784.726580:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.726737:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361536 00000100:00100000:3.0:1789760784.726745:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361536 00000100:00100000:3.0:1789760784.726795:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 765972181987 time 766006821103 state 0x0 ref 1 00000100:00100000:3.0:1789760784.726810:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766007081519 time 766006821103 state 0x1 ref 1 00000100:00100000:3.0:1789760784.726828:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766007081519 time 766042133177 state 0x0 ref 1 00000100:00100000:3.0:1789760784.726837:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507361536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.726886:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23199744, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.726894:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.726907:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760784.726911:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.726916:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.726926:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.755116:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00002000:00010000:2.0:1789760784.755218:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:2.0:1789760784.755234:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755248:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755257:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755264:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760784.755269:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755284:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755304:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.755312:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b971b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.755408:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507361536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28565us (28834us total) trans 4294972984 rc 0/0 00000100:00100000:2.0:1789760784.755418:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5694 00000100:00100000:2.0:1789760784.755427:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766007081519 time 766042133177 state 0x2 ref 0 00000100:00100000:2.0:1789760784.758427:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760784.758523:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361664 00000100:00100000:1.0:1789760784.758533:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361664 00000100:00100000:1.0:1789760784.758587:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766007081519 time 766042133177 state 0x0 ref 1 00000100:00100000:1.0:1789760784.758602:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766042393593 time 766042133177 state 0x1 ref 1 00000100:00100000:1.0:1789760784.758635:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766042393593 time 766073938650 state 0x0 ref 1 00000100:00100000:1.0:1789760784.758647:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507361664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760784.758736:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23203840, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:1.0:1789760784.758745:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.758761:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760784.758766:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.758773:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.758787:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.795009:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00002000:00010000:0.0:1789760784.795274:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:0.0:1789760784.795289:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795308:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795318:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795325:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.795330:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795339:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795361:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.795369:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9722 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.795460:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507361664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36810us (37037us total) trans 4294972985 rc 0/0 00000100:00100000:0.0:1789760784.795479:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5695 00000100:00100000:0.0:1789760784.795489:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766042393593 time 766073938650 state 0x2 ref 0 00000100:00100000:1.0:1789760784.803362:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760784.803446:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361792 00000100:00100000:3.0:1789760784.803457:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361792 00000100:00100000:3.0:1789760784.803744:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766042393593 time 766073938650 state 0x0 ref 1 00000100:00100000:3.0:1789760784.803758:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766074199066 time 766073938650 state 0x1 ref 1 00000100:00100000:3.0:1789760784.803846:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766074199066 time 766119147617 state 0x0 ref 1 00000100:00100000:3.0:1789760784.803860:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507361792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.803918:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23207936, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.803929:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.803945:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760784.803949:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.803955:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.803969:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.859981:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:1.0:1789760784.860005:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860019:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860030:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860038:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760784.860043:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860059:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860080:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.860089:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9729 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.860285:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507361792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56420us (56924us total) trans 4294972986 rc 0/0 00000100:00100000:1.0:1789760784.860298:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5696 00000100:00100000:1.0:1789760784.860310:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766074199066 time 766119147617 state 0x2 ref 0 00000001:00080000:1.0:1789760784.860359:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00000100:00100000:3.0:1789760784.869539:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760784.872980:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507361920 00000100:00100000:2.0:1789760784.872995:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507361920 00000100:00100000:2.0:1789760784.873062:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766074199066 time 766119147617 state 0x0 ref 1 00000100:00100000:2.0:1789760784.873078:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766119408033 time 766119147617 state 0x1 ref 1 00000100:00100000:2.0:1789760784.873102:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766119408033 time 766188405678 state 0x0 ref 1 00000100:00100000:2.0:1789760784.873114:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507361920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760784.876979:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23212032, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:3.0:1789760784.876995:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.877016:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760784.877023:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.877029:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.877041:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.936582:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:1.0:1789760784.936755:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:1.0:1789760784.936772:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936786:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936794:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936806:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760784.936811:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936820:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936842:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936847:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9730 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.938436:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507361920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65313us (68895us total) trans 4294972987 rc 0/0 00000100:00100000:1.0:1789760784.938449:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5697 00000100:00100000:1.0:1789760784.938475:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766119408033 time 766188405678 state 0x2 ref 0 00000100:00100000:2.0:1789760784.944173:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760784.944328:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362048 00000100:00100000:0.0:1789760784.944339:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362048 00000100:00100000:0.0:1789760784.944392:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766119408033 time 766188405678 state 0x0 ref 1 00000100:00100000:0.0:1789760784.944407:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766188666094 time 766188405678 state 0x1 ref 1 00000100:00100000:0.0:1789760784.944437:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766188666094 time 766259740850 state 0x0 ref 1 00000100:00100000:0.0:1789760784.944449:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507362048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760784.944508:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23216128, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:0.0:1789760784.944517:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.944532:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760784.944537:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.944543:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.944567:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.988671:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00002000:00010000:0.0:1789760784.988957:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:0.0:1789760784.988970:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.988982:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.988995:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.989002:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760784.989007:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.989015:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.989033:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.989039:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9737 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.989251:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507362048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44700us (44985us total) trans 4294972988 rc 0/0 00000100:00100000:0.0:1789760784.989261:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5698 00000100:00100000:0.0:1789760784.989271:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766188666094 time 766259740850 state 0x2 ref 0 00000100:00100000:3.0:1789760784.993109:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760784.993202:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362176 00000100:00100000:1.0:1789760784.993212:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362176 00000100:00100000:1.0:1789760784.993263:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766188666094 time 766259740850 state 0x0 ref 1 00000100:00100000:1.0:1789760784.993277:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766260001266 time 766259740850 state 0x1 ref 1 00000100:00100000:1.0:1789760784.993308:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766260001266 time 766308611610 state 0x0 ref 1 00000100:00100000:1.0:1789760784.993325:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507362176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760784.993373:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23220224, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xb807 00010000:00010000:1.0:1789760784.993382:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.993397:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760784.993402:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.993408:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.993421:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.041017:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:3.0:1789760785.041117:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23220224 -> 23224320 00002000:00010000:3.0:1789760785.041122:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:3.0:1789760785.041124:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:3.0:1789760785.041138:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041151:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041161:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041168:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760785.041173:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041187:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041206:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041214:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b973e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.041309:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507362176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47982us (48204us total) trans 4294972989 rc 0/0 00000100:00100000:3.0:1789760785.041319:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5699 00000100:00100000:3.0:1789760785.041329:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766260001266 time 766308611610 state 0x2 ref 0 00000100:00100000:3.0:1789760785.045671:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760785.045883:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362304 00000100:00100000:1.0:1789760785.045893:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362304 00000100:00100000:1.0:1789760785.045976:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766260001266 time 766308611610 state 0x0 ref 1 00000100:00100000:1.0:1789760785.045993:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766308872026 time 766308611610 state 0x1 ref 1 00000100:00100000:1.0:1789760785.046026:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766308872026 time 766361328858 state 0x0 ref 1 00000100:00100000:1.0:1789760785.046039:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760785.046086:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23224320, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:1.0:1789760785.046095:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.046116:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760785.046122:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.046128:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.046143:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760785.046975:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760785.099711:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00002000:00010000:1.0:1789760785.099874:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23224320 -> 23228416 00010000:00010000:1.0:1789760785.099889:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.099902:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.099980:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.099988:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760785.099992:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.100002:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.100021:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.100029:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9745 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.100119:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 54078us (54452us total) trans 4294972990 rc 0/0 00000100:00100000:1.0:1789760785.100129:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5700 00000100:00100000:1.0:1789760785.100139:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766308872026 time 766361328858 state 0x2 ref 0 00000100:00100000:0.0:1789760785.105066:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760785.107025:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362432 00000100:00100000:0.0:1789760785.107041:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362432 00000100:00100000:0.0:1789760785.107104:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766308872026 time 766361328858 state 0x0 ref 1 00000100:00100000:0.0:1789760785.107122:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766361589274 time 766361328858 state 0x1 ref 1 00000100:00100000:0.0:1789760785.107160:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766361589274 time 766422462868 state 0x0 ref 1 00000100:00100000:0.0:1789760785.107174:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507362432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760785.107224:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23228416, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:0.0:1789760785.107234:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.107251:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760785.107255:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.107262:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.107275:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.142732:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00002000:00010000:2.0:1789760785.142843:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:2.0:1789760785.142858:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142870:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142879:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142885:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.142890:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142899:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142919:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.142927:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b974c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.143056:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507362432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35880us (37994us total) trans 4294972991 rc 0/0 00000100:00100000:2.0:1789760785.143066:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5701 00000100:00100000:2.0:1789760785.143076:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766361589274 time 766422462868 state 0x2 ref 0 00000100:00100000:0.0:1789760785.150240:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.150334:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362560 00000100:00100000:3.0:1789760785.150343:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362560 00000100:00100000:3.0:1789760785.150395:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766361589274 time 766422462868 state 0x0 ref 1 00000100:00100000:3.0:1789760785.150411:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766422723284 time 766422462868 state 0x1 ref 1 00000100:00100000:3.0:1789760785.150445:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766422723284 time 766465748496 state 0x0 ref 1 00000100:00100000:3.0:1789760785.150458:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507362560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.150582:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23232512, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.150904:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.150928:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760785.150932:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.150938:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.150953:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.192509:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00002000:00010000:3.0:1789760785.192721:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:3.0:1789760785.192736:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192750:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192759:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192767:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760785.192772:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192780:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192801:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.192810:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9753 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.192889:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507362560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42428us (42654us total) trans 4294972992 rc 0/0 00000100:00100000:3.0:1789760785.192899:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5702 00000100:00100000:3.0:1789760785.192909:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766422723284 time 766465748496 state 0x2 ref 0 00000100:00100000:3.0:1789760785.198362:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760785.198613:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362688 00000100:00100000:1.0:1789760785.198623:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362688 00000100:00100000:1.0:1789760785.198682:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766422723284 time 766465748496 state 0x0 ref 1 00000100:00100000:1.0:1789760785.198736:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766466008912 time 766465748496 state 0x1 ref 1 00000100:00100000:1.0:1789760785.198759:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766466008912 time 766514062701 state 0x0 ref 1 00000100:00100000:1.0:1789760785.198772:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760785.198822:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23236608, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:1.0:1789760785.198831:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.198847:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760785.198851:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.198857:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.198877:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.245643:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00002000:00010000:2.0:1789760785.245781:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:2.0:1789760785.245798:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245811:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245822:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245829:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.245834:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245843:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245862:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.245871:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b975a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.245960:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47185us (47603us total) trans 4294972993 rc 0/0 00000100:00100000:2.0:1789760785.245974:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5703 00000100:00100000:2.0:1789760785.245984:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766466008912 time 766514062701 state 0x2 ref 0 00000100:00100000:0.0:1789760785.250329:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.250597:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362816 00000100:00100000:2.0:1789760785.250606:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362816 00000100:00100000:2.0:1789760785.250661:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766466008912 time 766514062701 state 0x0 ref 1 00000100:00100000:2.0:1789760785.250676:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766514323117 time 766514062701 state 0x1 ref 1 00000100:00100000:2.0:1789760785.250730:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766514323117 time 766566033363 state 0x0 ref 1 00000100:00100000:2.0:1789760785.250753:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507362816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.250801:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23240704, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.250810:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.250826:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760785.250830:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.250837:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.250850:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.293062:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00002000:00010000:2.0:1789760785.293213:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:2.0:1789760785.293228:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293242:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293253:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293261:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.293265:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293274:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293294:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.293302:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9761 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.293473:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507362816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42708us (43138us total) trans 4294972994 rc 0/0 00000100:00100000:2.0:1789760785.293485:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5704 00000100:00100000:2.0:1789760785.293494:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766514323117 time 766566033363 state 0x2 ref 0 00000100:00100000:0.0:1789760785.296895:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.297021:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507362944 00000100:00100000:3.0:1789760785.297031:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507362944 00000100:00100000:3.0:1789760785.297085:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766514323117 time 766566033363 state 0x0 ref 1 00000100:00100000:3.0:1789760785.297101:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766566293779 time 766566033363 state 0x1 ref 1 00000100:00100000:3.0:1789760785.297168:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766566293779 time 766612471158 state 0x0 ref 1 00000100:00100000:3.0:1789760785.297181:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.297228:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23244800, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.297237:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.297257:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760785.297262:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.297269:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.297283:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.307932:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760785.379665:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00002000:00010000:3.0:1789760785.379799:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23244800 -> 23248896 00010000:00010000:3.0:1789760785.379815:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379827:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379835:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379842:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760785.379846:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379855:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379876:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.379884:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9768 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.380581:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507362944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 83396us (83689us total) trans 4294972995 rc 0/0 00000100:00100000:3.0:1789760785.380593:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5705 00000100:00100000:3.0:1789760785.380602:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766566293779 time 766612471158 state 0x2 ref 0 00000100:00100000:0.0:1789760785.390176:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.390479:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363072 00000100:00100000:2.0:1789760785.390489:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363072 00000100:00100000:2.0:1789760785.390661:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766566293779 time 766612471158 state 0x0 ref 1 00000100:00100000:2.0:1789760785.390676:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766612731574 time 766612471158 state 0x1 ref 1 00000100:00100000:2.0:1789760785.390783:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766612731574 time 766706085496 state 0x0 ref 1 00000100:00100000:2.0:1789760785.390797:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.390847:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23248896, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.390856:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.390875:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760785.390880:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.390885:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.390900:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.430132:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00002000:00010000:0.0:1789760785.430209:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23248896 -> 23252992 00010000:00010000:0.0:1789760785.430225:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430238:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430246:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430254:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760785.430259:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430267:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430286:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.430293:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b976f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.430415:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39615us (40243us total) trans 4294972996 rc 0/0 00000100:00100000:0.0:1789760785.430425:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5706 00000100:00100000:0.0:1789760785.430435:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766612731574 time 766706085496 state 0x2 ref 0 00000100:00100000:0.0:1789760785.443807:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.444074:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363200 00000100:00100000:2.0:1789760785.444084:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363200 00000100:00100000:2.0:1789760785.444138:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766612731574 time 766706085496 state 0x0 ref 1 00000100:00100000:2.0:1789760785.444153:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766706345912 time 766706085496 state 0x1 ref 1 00000100:00100000:2.0:1789760785.444183:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766706345912 time 766759486755 state 0x0 ref 1 00000100:00100000:2.0:1789760785.444198:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.444244:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23252992, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.444254:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.444270:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760785.444274:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.444280:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.444295:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.475451:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00002000:00010000:0.0:1789760785.475513:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:0.0:1789760785.475528:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475540:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475548:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475556:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760785.475561:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475569:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475594:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.475602:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9776 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.475741:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31496us (31895us total) trans 4294972997 rc 0/0 00000100:00100000:0.0:1789760785.475755:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5707 00000100:00100000:0.0:1789760785.475765:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766706345912 time 766759486755 state 0x2 ref 0 00000100:00100000:1.0:1789760785.479162:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.479240:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363328 00000100:00100000:3.0:1789760785.479249:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363328 00000100:00100000:3.0:1789760785.479297:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766706345912 time 766759486755 state 0x0 ref 1 00000100:00100000:3.0:1789760785.479311:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766759747171 time 766759486755 state 0x1 ref 1 00000100:00100000:3.0:1789760785.479341:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766759747171 time 766794645153 state 0x0 ref 1 00000100:00100000:3.0:1789760785.479352:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.479394:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23257088, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.479403:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.479417:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760785.479421:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.479427:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.479446:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.509119:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00002000:00010000:2.0:1789760785.509247:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:2.0:1789760785.509263:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509276:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509286:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509294:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.509299:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509309:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509340:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.509350:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b977d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.509491:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30127us (30323us total) trans 4294972998 rc 0/0 00000100:00100000:2.0:1789760785.509599:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5708 00000100:00100000:2.0:1789760785.509609:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766759747171 time 766794645153 state 0x2 ref 0 00000100:00100000:1.0:1789760785.514056:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760785.514146:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363456 00000100:00100000:0.0:1789760785.514157:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363456 00000100:00100000:0.0:1789760785.514212:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766759747171 time 766794645153 state 0x0 ref 1 00000100:00100000:0.0:1789760785.514227:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766794905569 time 766794645153 state 0x1 ref 1 00000100:00100000:0.0:1789760785.514292:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766794905569 time 766829594835 state 0x0 ref 1 00000100:00100000:0.0:1789760785.514306:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760785.514351:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23261184, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:0.0:1789760785.514359:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.514374:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760785.514379:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.514385:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.514401:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.583181:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00002000:00010000:2.0:1789760785.583530:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:2.0:1789760785.583545:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583558:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583566:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583574:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.583579:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583587:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583606:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583614:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9784 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.583741:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 69433us (69689us total) trans 4294972999 rc 0/0 00000100:00100000:2.0:1789760785.583751:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5709 00000100:00100000:2.0:1789760785.583762:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766794905569 time 766829594835 state 0x2 ref 0 00000100:00100000:1.0:1789760785.589746:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760785.591105:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363584 00000100:00100000:0.0:1789760785.591118:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363584 00000100:00100000:0.0:1789760785.591178:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766794905569 time 766829594835 state 0x0 ref 1 00000100:00100000:0.0:1789760785.591192:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766829855251 time 766829594835 state 0x1 ref 1 00000100:00100000:0.0:1789760785.591223:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766829855251 time 766906526787 state 0x0 ref 1 00000100:00100000:0.0:1789760785.591235:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760785.591281:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23265280, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:0.0:1789760785.591289:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.591304:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760785.591309:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.591314:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.591328:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.630689:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c105821880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538294528:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760785.630777:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760785.631241:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294528 00000100:00100000:2.0:1789760785.631249:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294528 00000100:00100000:2.0:1789760785.631415:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1789760785.631427:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538294528:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760785.631448:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3834642432 avail 3814719488 00000100:00100000:1.0:1789760785.632055:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538294656:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760785.632583:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760785.632839:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294656 00000100:00100000:0.0:1789760785.632846:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294656 00000100:00100000:0.0:1789760785.632877:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760785.632885:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538294656:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760785.632913:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3834642432 avail 3814719488 00000100:00100000:0.0:1789760785.632962:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3462:x1876699538294656:12345-0@lo:13:osp-pre-0-1.0 Request processed in 78us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760785.632970:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789760785.633114:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3462:1876699538294656:0@lo:13:osp-pre-0-1.0 00000001:00080000:1.0:1789760785.636122:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00002000:00010000:0.0:1789760785.636299:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:0.0:1789760785.636314:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636326:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636335:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636342:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760785.636346:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636354:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636373:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.636381:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b978b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.636478:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45240us (46737us total) trans 4294973000 rc 0/0 00000100:00100000:0.0:1789760785.636502:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5710 00000100:00100000:0.0:1789760785.636512:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766829855251 time 766906526787 state 0x2 ref 0 00000100:00100000:2.0:1789760785.639507:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538294528:12345-0@lo:13:osp-pre-0-0.0 Request processed in 8074us (8721us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760785.639522:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789760785.640209:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c105821880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538294528:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760785.641259:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.641389:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363712 00000100:00100000:2.0:1789760785.641397:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363712 00000100:00100000:2.0:1789760785.641447:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766829855251 time 766906526787 state 0x0 ref 1 00000100:00100000:2.0:1789760785.641472:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766906787203 time 766906526787 state 0x1 ref 1 00000100:00100000:2.0:1789760785.641507:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766906787203 time 766956809745 state 0x0 ref 1 00000100:00100000:2.0:1789760785.641519:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.641565:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23269376, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.641574:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.641590:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760785.641595:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.641601:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.641616:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.682231:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00002000:00010000:2.0:1789760785.682322:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23269376 -> 23273472 00010000:00010000:2.0:1789760785.682338:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682352:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682361:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682369:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.682374:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682383:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682402:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.682411:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9792 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.682648:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41126us (41398us total) trans 4294973001 rc 0/0 00000100:00100000:2.0:1789760785.682659:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5711 00000100:00100000:2.0:1789760785.682669:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766906787203 time 766956809745 state 0x2 ref 0 00000100:00100000:0.0:1789760785.693050:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.693169:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363840 00000100:00100000:3.0:1789760785.693179:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363840 00000100:00100000:3.0:1789760785.693231:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766906787203 time 766956809745 state 0x0 ref 1 00000100:00100000:3.0:1789760785.693246:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766957070161 time 766956809745 state 0x1 ref 1 00000100:00100000:3.0:1789760785.693302:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766957070161 time 767008605490 state 0x0 ref 1 00000100:00100000:3.0:1789760785.693315:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.693362:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23273472, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.693372:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.693386:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760785.693391:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.693396:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.693412:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.725845:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00002000:00010000:3.0:1789760785.725920:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:3.0:1789760785.725936:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725948:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725957:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725964:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760785.725970:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725979:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725999:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.726007:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9799 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.726107:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507363840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32788us (33060us total) trans 4294973002 rc 0/0 00000100:00100000:3.0:1789760785.726117:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5712 00000100:00100000:3.0:1789760785.726126:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766957070161 time 767008605490 state 0x2 ref 0 00000100:00100000:0.0:1789760785.730226:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.731794:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507363968 00000100:00100000:2.0:1789760785.731868:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507363968 00000100:00100000:2.0:1789760785.731937:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 766957070161 time 767008605490 state 0x0 ref 1 00000100:00100000:2.0:1789760785.731953:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767008865906 time 767008605490 state 0x1 ref 1 00000100:00100000:2.0:1789760785.731985:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767008865906 time 767047288984 state 0x0 ref 1 00000100:00100000:2.0:1789760785.732004:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.732058:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23277568, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.732067:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.732082:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760785.732087:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.732093:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.732105:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.763150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00002000:00010000:2.0:1789760785.763258:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:2.0:1789760785.763274:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763287:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763297:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763305:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760785.763310:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763319:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763338:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763346:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.763436:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507363968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31432us (33216us total) trans 4294973003 rc 0/0 00000100:00100000:2.0:1789760785.763447:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5713 00000100:00100000:2.0:1789760785.763466:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767008865906 time 767047288984 state 0x2 ref 0 00000100:00100000:0.0:1789760785.767514:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.767580:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364096 00000100:00100000:2.0:1789760785.767590:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364096 00000100:00100000:2.0:1789760785.767647:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767008865906 time 767047288984 state 0x0 ref 1 00000100:00100000:2.0:1789760785.767662:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767047549400 time 767047288984 state 0x1 ref 1 00000100:00100000:2.0:1789760785.767726:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767047549400 time 767082996493 state 0x0 ref 1 00000100:00100000:2.0:1789760785.767740:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.767787:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23281664, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.767796:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767812:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760785.767817:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.767824:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767837:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.795849:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00002000:00010000:3.0:1789760785.796180:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:3.0:1789760785.796195:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796208:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796218:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796223:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760785.796227:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796235:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796253:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.796260:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b97a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.796390:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28649us (28889us total) trans 4294973004 rc 0/0 00000100:00100000:3.0:1789760785.796401:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5714 00000100:00100000:3.0:1789760785.796412:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767047549400 time 767082996493 state 0x2 ref 0 00000100:00100000:1.0:1789760785.812511:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.812588:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364224 00000100:00100000:3.0:1789760785.812598:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364224 00000100:00100000:3.0:1789760785.812651:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767047549400 time 767082996493 state 0x0 ref 1 00000100:00100000:3.0:1789760785.812667:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767083256909 time 767082996493 state 0x1 ref 1 00000100:00100000:3.0:1789760785.812718:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767083256909 time 767128020541 state 0x0 ref 1 00000100:00100000:3.0:1789760785.812731:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507364224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.812780:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23285760, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.812789:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.812805:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760785.812809:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.812815:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.812831:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.868071:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00002000:00010000:0.0:1789760785.871337:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:0.0:1789760785.871356:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871370:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871379:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871386:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760785.871391:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871401:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871423:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.871430:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b97ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.871655:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507364224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58920us (59148us total) trans 4294973005 rc 0/0 00000100:00100000:0.0:1789760785.871667:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5715 00000100:00100000:0.0:1789760785.871677:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767083256909 time 767128020541 state 0x2 ref 0 00000100:00100000:0.0:1789760785.884263:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760785.884352:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364352 00000100:00100000:2.0:1789760785.884362:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364352 00000100:00100000:2.0:1789760785.884416:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767083256909 time 767128020541 state 0x0 ref 1 00000100:00100000:2.0:1789760785.884430:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767128280957 time 767128020541 state 0x1 ref 1 00000100:00100000:2.0:1789760785.884471:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767128280957 time 767199774017 state 0x0 ref 1 00000100:00100000:2.0:1789760785.884485:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760785.884542:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23289856, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:2.0:1789760785.884552:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.884567:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760785.884571:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.884578:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.884596:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.949941:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00002000:00010000:1.0:1789760785.950227:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:1.0:1789760785.950243:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950256:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950266:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950274:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760785.950279:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950288:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950306:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.950314:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b97b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.950440:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65953us (66181us total) trans 4294973006 rc 0/0 00000100:00100000:1.0:1789760785.950451:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5716 00000100:00100000:1.0:1789760785.950470:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767128280957 time 767199774017 state 0x2 ref 0 00000100:00100000:1.0:1789760785.954655:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760785.957853:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364480 00000100:00100000:3.0:1789760785.957869:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364480 00000100:00100000:3.0:1789760785.957935:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767128280957 time 767199774017 state 0x0 ref 1 00000100:00100000:3.0:1789760785.957954:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767200034433 time 767199774017 state 0x1 ref 1 00000100:00100000:3.0:1789760785.958131:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767200034433 time 767273431822 state 0x0 ref 1 00000100:00100000:3.0:1789760785.958146:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507364480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760785.958194:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23293952, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:3.0:1789760785.958203:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.958218:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760785.958223:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.958229:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.958244:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.002811:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00002000:00010000:3.0:1789760786.002950:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:3.0:1789760786.002965:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.002977:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.002986:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.002999:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.003004:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.003012:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.003032:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.003040:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.003168:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507364480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45020us (48516us total) trans 4294973007 rc 0/0 00000100:00100000:3.0:1789760786.003176:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5717 00000100:00100000:3.0:1789760786.003185:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767200034433 time 767273431822 state 0x2 ref 0 00000100:00100000:0.0:1789760786.010449:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.010658:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364608 00000100:00100000:3.0:1789760786.010669:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364608 00000100:00100000:3.0:1789760786.010752:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767200034433 time 767273431822 state 0x0 ref 1 00000100:00100000:3.0:1789760786.010767:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767273692238 time 767273431822 state 0x1 ref 1 00000100:00100000:0.0:1789760786.010919:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767273692238 time 767326220190 state 0x0 ref 1 00000100:00100000:0.0:1789760786.010934:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507364608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760786.010982:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23298048, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xb807 00010000:00010000:0.0:1789760786.010992:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.011008:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760786.011013:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.011020:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.011040:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.040297:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00002000:00010000:3.0:1789760786.040408:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23298048 -> 23302144 00002000:00010000:3.0:1789760786.040412:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:3.0:1789760786.040416:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:3.0:1789760786.040428:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040441:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040452:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040471:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.040476:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040486:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040512:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.040520:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b97c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.040612:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507364608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29677us (30168us total) trans 4294973008 rc 0/0 00000100:00100000:3.0:1789760786.040623:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5718 00000100:00100000:3.0:1789760786.040633:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767273692238 time 767326220190 state 0x2 ref 0 00000100:00100000:3.0:1789760786.044024:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.044302:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364736 00000100:00100000:1.0:1789760786.044313:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364736 00000100:00100000:1.0:1789760786.044366:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767273692238 time 767326220190 state 0x0 ref 1 00000100:00100000:1.0:1789760786.044382:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767326480606 time 767326220190 state 0x1 ref 1 00000100:00100000:1.0:1789760786.044415:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767326480606 time 767359718099 state 0x0 ref 1 00000100:00100000:1.0:1789760786.044427:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.044484:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23302144, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.044848:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.044866:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.044870:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.044877:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.044892:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.077513:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00002000:00010000:3.0:1789760786.077664:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23302144 -> 23306240 00010000:00010000:3.0:1789760786.077681:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077733:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077744:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077752:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.077758:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077769:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077794:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.077802:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b97ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.077891:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33459us (33869us total) trans 4294973009 rc 0/0 00000100:00100000:3.0:1789760786.077906:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5719 00000100:00100000:3.0:1789760786.077916:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767326480606 time 767359718099 state 0x2 ref 0 00000100:00100000:1.0:1789760786.081049:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.081127:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364864 00000100:00100000:3.0:1789760786.081137:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364864 00000100:00100000:3.0:1789760786.081184:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767326480606 time 767359718099 state 0x0 ref 1 00000100:00100000:3.0:1789760786.081198:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767359978515 time 767359718099 state 0x1 ref 1 00000100:00100000:3.0:1789760786.081227:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767359978515 time 767396530482 state 0x0 ref 1 00000100:00100000:3.0:1789760786.081238:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507364864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.081284:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23306240, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.081293:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.081309:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.081314:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.081321:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.081335:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.113790:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:3.0:1789760786.113811:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113825:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113834:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113842:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.113847:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113857:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113878:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.113886:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.113989:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507364864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32747us (32943us total) trans 4294973010 rc 0/0 00000100:00100000:3.0:1789760786.114000:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5720 00000100:00100000:3.0:1789760786.114010:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767359978515 time 767396530482 state 0x2 ref 0 00000100:00100000:1.0:1789760786.117424:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.117543:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507364992 00000100:00100000:3.0:1789760786.117553:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507364992 00000100:00100000:3.0:1789760786.117603:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767359978515 time 767396530482 state 0x0 ref 1 00000100:00100000:3.0:1789760786.117619:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767396790898 time 767396530482 state 0x1 ref 1 00000100:00100000:3.0:1789760786.117649:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767396790898 time 767432952607 state 0x0 ref 1 00000100:00100000:3.0:1789760786.117661:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507364992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.117730:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23310336, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.117740:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.117755:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.117759:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.117766:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.117780:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.118442:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00000100:00100000:1.0:1789760786.141394:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538294784:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760786.141479:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760786.141569:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294784 00000100:00100000:1.0:1789760786.141576:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294784 00000100:00100000:1.0:1789760786.141617:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789760786.141625:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538294784:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760786.141663:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760786.141769:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3463:x1876699538294784:12345-0@lo:41:osp-pre-1-0.0 Request processed in 143us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760786.141776:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789760786.143482:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3463:1876699538294784:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760786.160930:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00002000:00010000:3.0:1789760786.161087:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:3.0:1789760786.161103:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161116:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161130:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161138:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.161143:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161152:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161172:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.161180:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.161302:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507364992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43638us (43883us total) trans 4294973011 rc 0/0 00000100:00100000:3.0:1789760786.161313:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5721 00000100:00100000:3.0:1789760786.161322:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767396790898 time 767432952607 state 0x2 ref 0 00000100:00100000:1.0:1789760786.164421:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.164565:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365120 00000100:00100000:3.0:1789760786.164575:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365120 00000100:00100000:3.0:1789760786.164627:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767396790898 time 767432952607 state 0x0 ref 1 00000100:00100000:3.0:1789760786.164642:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767433213023 time 767432952607 state 0x1 ref 1 00000100:00100000:3.0:1789760786.164731:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767433213023 time 767480033906 state 0x0 ref 1 00000100:00100000:3.0:1789760786.164744:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.164791:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23314432, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.164801:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.164815:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.164819:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.164825:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.164838:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.188484:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00002000:00010000:1.0:1789760786.188662:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23314432 -> 23318528 00010000:00010000:1.0:1789760786.188677:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188689:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188726:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188734:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.188739:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188754:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188774:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.188783:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b97df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.188873:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24125us (24458us total) trans 4294973012 rc 0/0 00000100:00100000:1.0:1789760786.188882:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5722 00000100:00100000:1.0:1789760786.188894:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767433213023 time 767480033906 state 0x2 ref 0 00000100:00100000:1.0:1789760786.192464:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.192944:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365248 00000100:00100000:3.0:1789760786.192954:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365248 00000100:00100000:3.0:1789760786.193015:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767433213023 time 767480033906 state 0x0 ref 1 00000100:00100000:3.0:1789760786.193030:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767480294322 time 767480033906 state 0x1 ref 1 00000100:00100000:3.0:1789760786.193060:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767480294322 time 767508363887 state 0x0 ref 1 00000100:00100000:3.0:1789760786.193072:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507365248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.193119:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23318528, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.193128:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.193144:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.193148:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.193155:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.193170:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.220747:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00002000:00010000:1.0:1789760786.220870:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23318528 -> 23322624 00010000:00010000:1.0:1789760786.220886:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220909:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220920:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220933:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.220939:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220948:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220969:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.220978:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b97e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.221070:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507365248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27995us (28621us total) trans 4294973013 rc 0/0 00000100:00100000:1.0:1789760786.221081:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5723 00000100:00100000:1.0:1789760786.221091:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767480294322 time 767508363887 state 0x2 ref 0 00000100:00100000:2.0:1789760786.225379:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.225553:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365376 00000100:00100000:3.0:1789760786.225563:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365376 00000100:00100000:3.0:1789760786.225618:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767480294322 time 767508363887 state 0x0 ref 1 00000100:00100000:3.0:1789760786.225634:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767508624303 time 767508363887 state 0x1 ref 1 00000100:00100000:3.0:1789760786.225851:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767508624303 time 767541152391 state 0x0 ref 1 00000100:00100000:3.0:1789760786.225866:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.225994:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23322624, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.226004:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.226019:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.226023:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.226029:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.226044:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.253815:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00002000:00010000:3.0:1789760786.253962:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:3.0:1789760786.253979:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.253992:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.254001:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.254009:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.254014:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.254024:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.254043:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.254052:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.254183:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28314us (28809us total) trans 4294973014 rc 0/0 00000100:00100000:3.0:1789760786.254193:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5724 00000100:00100000:3.0:1789760786.254202:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767508624303 time 767541152391 state 0x2 ref 0 00000100:00100000:0.0:1789760786.258589:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.258681:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365504 00000100:00100000:1.0:1789760786.258757:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365504 00000100:00100000:1.0:1789760786.258812:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767508624303 time 767541152391 state 0x0 ref 1 00000100:00100000:1.0:1789760786.258828:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767541412807 time 767541152391 state 0x1 ref 1 00000100:00100000:1.0:1789760786.258857:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767541412807 time 767574160490 state 0x0 ref 1 00000100:00100000:1.0:1789760786.258871:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507365504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.258917:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23326720, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.258926:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.258942:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.258946:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.258952:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.258966:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.308449:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23326720 -> 23330816 00010000:00010000:1.0:1789760786.308477:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308501:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308511:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308519:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.308524:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308534:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308553:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.308561:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b97f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.308648:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507365504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49776us (50065us total) trans 4294973015 rc 0/0 00000100:00100000:1.0:1789760786.308660:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5725 00000100:00100000:1.0:1789760786.308668:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767541412807 time 767574160490 state 0x2 ref 0 00000001:00080000:3.0:1789760786.308895:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00000100:00100000:1.0:1789760786.312297:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.312483:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365632 00000100:00100000:3.0:1789760786.312503:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365632 00000100:00100000:3.0:1789760786.312556:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767541412807 time 767574160490 state 0x0 ref 1 00000100:00100000:3.0:1789760786.312571:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767574420906 time 767574160490 state 0x1 ref 1 00000100:00100000:3.0:1789760786.312602:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767574420906 time 767627905335 state 0x0 ref 1 00000100:00100000:3.0:1789760786.312615:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.312661:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23330816, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.312670:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.312686:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.312719:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.312727:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.312741:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.331856:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760786.340337:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00002000:00010000:3.0:1789760786.340443:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:3.0:1789760786.340469:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340483:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340610:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340619:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.340624:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340634:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340655:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.340663:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b97fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.340871:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28252us (28579us total) trans 4294973016 rc 0/0 00000100:00100000:3.0:1789760786.340886:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5726 00000100:00100000:3.0:1789760786.340961:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767574420906 time 767627905335 state 0x2 ref 0 00000100:00100000:2.0:1789760786.345115:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.345187:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365760 00000100:00100000:3.0:1789760786.345196:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365760 00000100:00100000:3.0:1789760786.345249:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767574420906 time 767627905335 state 0x0 ref 1 00000100:00100000:3.0:1789760786.345263:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767628165751 time 767627905335 state 0x1 ref 1 00000100:00100000:3.0:1789760786.345291:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767628165751 time 767660594866 state 0x0 ref 1 00000100:00100000:3.0:1789760786.345304:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507365760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.345352:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23334912, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.345360:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.345374:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.345379:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.345385:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.345399:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.373112:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00002000:00010000:3.0:1789760786.373228:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:3.0:1789760786.373245:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373259:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373269:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373276:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.373281:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373291:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373311:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.373320:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9802 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.373418:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507365760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28111us (28308us total) trans 4294973017 rc 0/0 00000100:00100000:3.0:1789760786.373428:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5727 00000100:00100000:3.0:1789760786.373439:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767628165751 time 767660594866 state 0x2 ref 0 00000100:00100000:3.0:1789760786.376785:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.376909:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507365888 00000100:00100000:1.0:1789760786.376918:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507365888 00000100:00100000:1.0:1789760786.376969:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767628165751 time 767660594866 state 0x0 ref 1 00000100:00100000:1.0:1789760786.376984:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767660855282 time 767660594866 state 0x1 ref 1 00000100:00100000:1.0:1789760786.377016:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767660855282 time 767692319092 state 0x0 ref 1 00000100:00100000:1.0:1789760786.377029:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507365888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.377079:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23339008, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.377089:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.377105:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.377110:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.377117:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.377131:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.410477:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23339008 -> 23343104 00010000:00010000:1.0:1789760786.410505:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410518:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410527:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410534:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.410539:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410547:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410567:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.410575:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9809 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.410662:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507365888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33630us (33882us total) trans 4294973018 rc 0/0 00000100:00100000:1.0:1789760786.410672:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5728 00000100:00100000:1.0:1789760786.410682:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767660855282 time 767692319092 state 0x2 ref 0 00000001:00080000:0.0:1789760786.412036:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00000100:00100000:1.0:1789760786.415010:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.415122:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366016 00000100:00100000:3.0:1789760786.415131:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366016 00000100:00100000:3.0:1789760786.415178:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767660855282 time 767692319092 state 0x0 ref 1 00000100:00100000:3.0:1789760786.415193:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767692579508 time 767692319092 state 0x1 ref 1 00000100:00100000:3.0:1789760786.415249:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767692579508 time 767730552337 state 0x0 ref 1 00000100:00100000:3.0:1789760786.415261:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.415305:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23343104, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.415313:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.415328:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.415332:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.415338:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.415351:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.450853:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00002000:00010000:1.0:1789760786.451013:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:1.0:1789760786.451028:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451041:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451050:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451058:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.451063:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451079:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451098:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.451106:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9810 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.451199:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35934us (36192us total) trans 4294973019 rc 0/0 00000100:00100000:1.0:1789760786.451209:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5729 00000100:00100000:1.0:1789760786.451220:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767692579508 time 767730552337 state 0x2 ref 0 00000100:00100000:2.0:1789760786.454447:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.454631:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366144 00000100:00100000:3.0:1789760786.454640:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366144 00000100:00100000:3.0:1789760786.454728:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767692579508 time 767730552337 state 0x0 ref 1 00000100:00100000:3.0:1789760786.454744:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767730812753 time 767730552337 state 0x1 ref 1 00000100:00100000:3.0:1789760786.454772:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767730812753 time 767770074685 state 0x0 ref 1 00000100:00100000:3.0:1789760786.454785:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.454832:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23347200, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.454840:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.454857:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.454862:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.454868:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.454883:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.485002:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:1.0:1789760786.485271:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:1.0:1789760786.485287:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485300:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485309:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485317:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.485322:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485332:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485352:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.485360:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9817 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.485471:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30682us (31028us total) trans 4294973020 rc 0/0 00000100:00100000:1.0:1789760786.485481:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5730 00000100:00100000:1.0:1789760786.486383:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767730812753 time 767770074685 state 0x2 ref 0 00000100:00100000:3.0:1789760786.490244:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.490331:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366272 00000100:00100000:1.0:1789760786.490340:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366272 00000100:00100000:1.0:1789760786.490395:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767730812753 time 767770074685 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490410:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767770335101 time 767770074685 state 0x1 ref 1 00000100:00100000:1.0:1789760786.490443:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767770335101 time 767805746146 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490465:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.490518:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23351296, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.490527:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490677:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.490682:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.490689:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490811:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.513406:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:1.0:1789760786.513425:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513437:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513447:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513464:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.513469:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513478:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513757:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.513767:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b981e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.513933:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507366272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23475us (23693us total) trans 4294973021 rc 0/0 00000100:00100000:1.0:1789760786.513944:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5731 00000100:00100000:1.0:1789760786.513954:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767770335101 time 767805746146 state 0x2 ref 0 00000001:00080000:1.0:1789760786.513992:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00000100:00100000:1.0:1789760786.519559:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760786.519644:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366400 00000100:00100000:3.0:1789760786.519652:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366400 00000100:00100000:3.0:1789760786.519729:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767770335101 time 767805746146 state 0x0 ref 1 00000100:00100000:3.0:1789760786.519744:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767806006562 time 767805746146 state 0x1 ref 1 00000100:00100000:3.0:1789760786.519773:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767806006562 time 767835076731 state 0x0 ref 1 00000100:00100000:3.0:1789760786.519785:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760786.519830:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23355392, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:3.0:1789760786.519839:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.519855:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760786.519859:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.519885:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.519901:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.567034:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00002000:00010000:3.0:1789760786.571651:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:3.0:1789760786.571675:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571689:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571724:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571731:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.571736:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571745:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571767:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.571775:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9825 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.571954:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52163us (52398us total) trans 4294973022 rc 0/0 00000100:00100000:2.0:1789760786.571966:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5732 00000100:00100000:2.0:1789760786.571977:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767806006562 time 767835076731 state 0x2 ref 0 00000100:00100000:2.0:1789760786.575464:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.576739:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366528 00000100:00100000:1.0:1789760786.576752:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366528 00000100:00100000:1.0:1789760786.576808:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767806006562 time 767835076731 state 0x0 ref 1 00000100:00100000:1.0:1789760786.576823:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767835337147 time 767835076731 state 0x1 ref 1 00000100:00100000:1.0:1789760786.576854:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767835337147 time 767892158032 state 0x0 ref 1 00000100:00100000:1.0:1789760786.576866:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.576914:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23359488, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.576923:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.576938:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.576942:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.576948:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.576961:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.625167:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00002000:00010000:1.0:1789760786.625237:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23359488 -> 23363584 00010000:00010000:1.0:1789760786.625254:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625268:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625277:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625283:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.625288:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625297:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625323:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625332:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b982c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.625429:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48559us (49976us total) trans 4294973023 rc 0/0 00000100:00100000:1.0:1789760786.625438:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5733 00000100:00100000:1.0:1789760786.625448:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767835337147 time 767892158032 state 0x2 ref 0 00000100:00100000:3.0:1789760786.631511:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.631610:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366656 00000100:00100000:1.0:1789760786.631619:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366656 00000100:00100000:1.0:1789760786.631676:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767835337147 time 767892158032 state 0x0 ref 1 00000100:00100000:1.0:1789760786.631724:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767892418448 time 767892158032 state 0x1 ref 1 00000100:00100000:1.0:1789760786.631754:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767892418448 time 767947057493 state 0x0 ref 1 00000100:00100000:1.0:1789760786.631767:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.631825:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23363584, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.631834:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631849:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760786.631854:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.631861:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631875:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.652934:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760786.652958:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c104068700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538294912:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760786.653023:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760786.653093:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538294912 00000100:00100000:3.0:1789760786.653100:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538294912 00000100:00100000:3.0:1789760786.653146:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789760786.653156:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538294912:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760786.653192:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760786.653240:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538294912:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760786.653246:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789760786.653292:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c104068700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538294912:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760786.665427:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00002000:00010000:1.0:1789760786.665505:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:1.0:1789760786.665521:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665533:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665542:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665549:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.665554:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665562:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665579:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.665591:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9833 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.665677:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33908us (34171us total) trans 4294973024 rc 0/0 00000100:00100000:1.0:1789760786.665686:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5734 00000100:00100000:1.0:1789760786.665730:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767892418448 time 767947057493 state 0x2 ref 0 00000100:00100000:3.0:1789760786.669409:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760786.669556:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366784 00000100:00100000:2.0:1789760786.669566:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366784 00000100:00100000:2.0:1789760786.669624:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767892418448 time 767947057493 state 0x0 ref 1 00000100:00100000:2.0:1789760786.669638:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767947317909 time 767947057493 state 0x1 ref 1 00000100:00100000:2.0:1789760786.670183:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767947317909 time 767985482671 state 0x0 ref 1 00000100:00100000:2.0:1789760786.670202:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760786.670253:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23367680, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760786.670262:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.670278:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760786.670282:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.670288:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.670301:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.696815:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00002000:00010000:2.0:1789760786.696968:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:2.0:1789760786.696988:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697003:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697024:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697032:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760786.697037:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697046:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697069:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.697077:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b983a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.697171:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507366784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26968us (27762us total) trans 4294973025 rc 0/0 00000100:00100000:2.0:1789760786.697182:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5735 00000100:00100000:2.0:1789760786.697191:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767947317909 time 767985482671 state 0x2 ref 0 00000100:00100000:3.0:1789760786.701860:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.702047:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507366912 00000100:00100000:1.0:1789760786.702056:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507366912 00000100:00100000:1.0:1789760786.702105:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767947317909 time 767985482671 state 0x0 ref 1 00000100:00100000:1.0:1789760786.702119:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767985743087 time 767985482671 state 0x1 ref 1 00000100:00100000:1.0:1789760786.702187:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767985743087 time 768017488972 state 0x0 ref 1 00000100:00100000:1.0:1789760786.702201:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507366912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760786.702247:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23371776, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:1.0:1789760786.702257:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.702275:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760786.702280:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.702286:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.702301:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.734889:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23371776 -> 23375872 00010000:00010000:1.0:1789760786.734909:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734922:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734932:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734940:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.734944:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734960:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734982:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.734990:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9841 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.735083:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507366912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32880us (33228us total) trans 4294973026 rc 0/0 00000100:00100000:1.0:1789760786.735094:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5736 00000100:00100000:1.0:1789760786.735103:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767985743087 time 768017488972 state 0x2 ref 0 00000001:00080000:1.0:1789760786.735144:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00000100:00100000:1.0:1789760786.738837:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760786.738935:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367040 00000100:00100000:2.0:1789760786.738945:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367040 00000100:00100000:2.0:1789760786.739000:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 767985743087 time 768017488972 state 0x0 ref 1 00000100:00100000:2.0:1789760786.739014:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768017749388 time 768017488972 state 0x1 ref 1 00000100:00100000:2.0:1789760786.739045:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768017749388 time 768054348160 state 0x0 ref 1 00000100:00100000:2.0:1789760786.739058:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507367040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760786.739105:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23375872, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760786.739114:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739130:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760786.739134:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.739140:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739153:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.777708:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00002000:00010000:1.0:1789760786.778299:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:1.0:1789760786.778315:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778329:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778339:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778346:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.778352:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778360:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778380:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.778388:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9848 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.778757:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507367040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39695us (39923us total) trans 4294973027 rc 0/0 00000100:00100000:1.0:1789760786.778769:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5737 00000100:00100000:1.0:1789760786.778779:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768017749388 time 768054348160 state 0x2 ref 0 00000100:00100000:1.0:1789760786.783572:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760786.783661:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367168 00000100:00100000:2.0:1789760786.783671:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367168 00000100:00100000:2.0:1789760786.783892:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768017749388 time 768054348160 state 0x0 ref 1 00000100:00100000:2.0:1789760786.783908:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768054608576 time 768054348160 state 0x1 ref 1 00000100:00100000:2.0:1789760786.784089:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768054608576 time 768099390568 state 0x0 ref 1 00000100:00100000:2.0:1789760786.784104:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507367168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760786.784148:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23379968, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760786.784157:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.784173:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760786.784178:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.784185:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.784204:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.827776:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00002000:00010000:3.0:1789760786.828095:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:3.0:1789760786.828110:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828124:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828139:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828146:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.828151:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828160:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828192:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.828201:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b984f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.828294:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507367168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44188us (44726us total) trans 4294973028 rc 0/0 00000100:00100000:3.0:1789760786.828303:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5738 00000100:00100000:3.0:1789760786.828313:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768054608576 time 768099390568 state 0x2 ref 0 00000100:00100000:3.0:1789760786.831522:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760786.831610:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367296 00000100:00100000:1.0:1789760786.831620:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367296 00000100:00100000:1.0:1789760786.831670:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768054608576 time 768099390568 state 0x0 ref 1 00000100:00100000:1.0:1789760786.831684:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768099650984 time 768099390568 state 0x1 ref 1 00000100:00100000:2.0:1789760786.831848:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768099650984 time 768147148083 state 0x0 ref 1 00000100:00100000:2.0:1789760786.831864:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507367296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760786.831913:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23384064, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760786.832084:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.832102:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760786.832106:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.832112:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.832127:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.880090:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00002000:00010000:1.0:1789760786.880180:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:1.0:1789760786.880195:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880208:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880217:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880224:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760786.880230:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880239:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880259:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.880266:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9856 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.880395:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507367296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48530us (48876us total) trans 4294973029 rc 0/0 00000100:00100000:1.0:1789760786.880406:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5739 00000100:00100000:1.0:1789760786.880415:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768099650984 time 768147148083 state 0x2 ref 0 00000100:00100000:1.0:1789760786.884365:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760786.884466:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367424 00000100:00100000:2.0:1789760786.884476:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367424 00000100:00100000:2.0:1789760786.884587:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768099650984 time 768147148083 state 0x0 ref 1 00000100:00100000:2.0:1789760786.884602:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768147408499 time 768147148083 state 0x1 ref 1 00000100:00100000:2.0:1789760786.884676:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768147408499 time 768199978565 state 0x0 ref 1 00000100:00100000:2.0:1789760786.884731:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760786.884785:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23388160, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760786.884795:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.884810:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760786.884815:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.884822:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.884836:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.929132:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00002000:00010000:3.0:1789760786.930108:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:3.0:1789760786.930125:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930138:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930148:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930155:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760786.930161:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930170:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930190:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.930199:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b985d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.930343:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45649us (45984us total) trans 4294973030 rc 0/0 00000100:00100000:3.0:1789760786.930355:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5740 00000100:00100000:3.0:1789760786.930365:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768147408499 time 768199978565 state 0x2 ref 0 00000100:00100000:3.0:1789760786.934019:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760786.934149:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367552 00000100:00100000:0.0:1789760786.934160:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367552 00000100:00100000:0.0:1789760786.934213:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768147408499 time 768199978565 state 0x0 ref 1 00000100:00100000:0.0:1789760786.934230:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768200238981 time 768199978565 state 0x1 ref 1 00000100:00100000:0.0:1789760786.934296:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768200238981 time 768249598661 state 0x0 ref 1 00000100:00100000:0.0:1789760786.934308:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507367552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760786.934353:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23392256, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:0.0:1789760786.934362:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.934377:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760786.934382:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.934388:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.934402:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.010279:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:0.0:1789760787.010434:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:0.0:1789760787.010460:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010481:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010568:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010576:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760787.010581:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010591:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010613:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.010621:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9864 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.010813:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507367552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 76501us (76800us total) trans 4294973031 rc 0/0 00000100:00100000:0.0:1789760787.010824:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5741 00000100:00100000:0.0:1789760787.010834:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768200238981 time 768249598661 state 0x2 ref 0 00000100:00100000:3.0:1789760787.014141:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760787.014240:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367680 00000100:00100000:2.0:1789760787.014251:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367680 00000100:00100000:2.0:1789760787.014323:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768200238981 time 768249598661 state 0x0 ref 1 00000100:00100000:2.0:1789760787.014338:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768249859077 time 768249598661 state 0x1 ref 1 00000100:00100000:2.0:1789760787.014369:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768249859077 time 768329672878 state 0x0 ref 1 00000100:00100000:2.0:1789760787.014382:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760787.014429:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23396352, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xb807 00010000:00010000:2.0:1789760787.014438:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.014469:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760787.014474:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.014491:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.014506:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760787.015582:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760787.105575:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:2.0:1789760787.105757:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23396352 -> 23400448 00002000:00010000:2.0:1789760787.105763:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760787.105766:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:2.0:1789760787.105791:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105804:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105813:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105822:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760787.105827:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105837:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105857:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.105866:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b986b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.106261:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 91877us (92126us total) trans 4294973032 rc 0/0 00000100:00100000:2.0:1789760787.106273:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5742 00000100:00100000:2.0:1789760787.106282:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768249859077 time 768329672878 state 0x2 ref 0 00000100:00100000:3.0:1789760787.109203:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760787.109278:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367808 00000100:00100000:2.0:1789760787.109287:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367808 00000100:00100000:2.0:1789760787.109340:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768249859077 time 768329672878 state 0x0 ref 1 00000100:00100000:2.0:1789760787.109356:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768329933294 time 768329672878 state 0x1 ref 1 00000100:00100000:2.0:1789760787.109389:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768329933294 time 768424692239 state 0x0 ref 1 00000100:00100000:2.0:1789760787.109403:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760787.109460:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23400448, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:2.0:1789760787.109469:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.109492:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760787.109499:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.109505:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.109520:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.147798:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00002000:00010000:2.0:1789760787.148018:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23400448 -> 23404544 00010000:00010000:2.0:1789760787.148034:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148048:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148058:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148065:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760787.148070:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148080:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148101:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.148116:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9872 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.148289:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38883us (39090us total) trans 4294973033 rc 0/0 00000100:00100000:2.0:1789760787.148301:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5743 00000100:00100000:2.0:1789760787.148311:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768329933294 time 768424692239 state 0x2 ref 0 00000100:00100000:0.0:1789760787.152415:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.152590:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507367936 00000100:00100000:3.0:1789760787.152599:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507367936 00000100:00100000:3.0:1789760787.152651:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768329933294 time 768424692239 state 0x0 ref 1 00000100:00100000:3.0:1789760787.152666:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768424952655 time 768424692239 state 0x1 ref 1 00000100:00100000:3.0:1789760787.152828:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768424952655 time 768468129245 state 0x0 ref 1 00000100:00100000:3.0:1789760787.152842:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.152901:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23404544, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.152972:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.152989:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.152994:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.153000:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.153014:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.185666:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00002000:00010000:2.0:1789760787.186016:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23404544 -> 23408640 00010000:00010000:2.0:1789760787.186033:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186046:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186057:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186065:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760787.186070:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186080:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186106:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.186115:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9879 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.186216:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507367936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33371us (33806us total) trans 4294973034 rc 0/0 00000100:00100000:2.0:1789760787.186228:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5744 00000100:00100000:2.0:1789760787.186237:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768424952655 time 768468129245 state 0x2 ref 0 00000100:00100000:0.0:1789760787.191128:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760787.191295:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368064 00000100:00100000:2.0:1789760787.191306:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368064 00000100:00100000:2.0:1789760787.191358:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768424952655 time 768468129245 state 0x0 ref 1 00000100:00100000:2.0:1789760787.191374:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768468389661 time 768468129245 state 0x1 ref 1 00000100:00100000:2.0:1789760787.191442:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768468389661 time 768506744488 state 0x0 ref 1 00000100:00100000:2.0:1789760787.191466:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507368064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760787.191524:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23408640, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:2.0:1789760787.191533:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.191548:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760787.191554:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.191560:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.191575:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760787.235409:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:3.0:1789760787.235428:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235459:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235469:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235476:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.235480:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235498:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235517:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.235523:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9880 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.235618:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507368064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44150us (44495us total) trans 4294973035 rc 0/0 00000100:00100000:3.0:1789760787.235628:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5745 00000100:00100000:3.0:1789760787.235638:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768468389661 time 768506744488 state 0x2 ref 0 00000100:00100000:3.0:1789760787.239169:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.239220:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368192 00000100:00100000:3.0:1789760787.239229:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368192 00000100:00100000:3.0:1789760787.239276:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768468389661 time 768506744488 state 0x0 ref 1 00000100:00100000:3.0:1789760787.239289:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768507004904 time 768506744488 state 0x1 ref 1 00000100:00100000:3.0:1789760787.239315:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768507004904 time 768554618938 state 0x0 ref 1 00000100:00100000:3.0:1789760787.239326:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507368192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.239368:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23412736, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.239376:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.239390:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.239395:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.239400:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.239415:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.240150:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00000001:00080000:1.0:1789760787.324003:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00002000:00010000:0.0:1789760787.324117:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:0.0:1789760787.324133:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324145:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324154:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324162:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760787.324166:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324176:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324208:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.324217:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9887 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.324373:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507368192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 85042us (85208us total) trans 4294973036 rc 0/0 00000100:00100000:0.0:1789760787.324385:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5746 00000100:00100000:0.0:1789760787.324396:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768507004904 time 768554618938 state 0x2 ref 0 00000100:00100000:2.0:1789760787.335831:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.336823:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368320 00000100:00100000:3.0:1789760787.336833:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368320 00000100:00100000:3.0:1789760787.336901:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768507004904 time 768554618938 state 0x0 ref 1 00000100:00100000:3.0:1789760787.336917:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768554879354 time 768554618938 state 0x1 ref 1 00000100:00100000:3.0:1789760787.336993:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768554879354 time 768652295772 state 0x0 ref 1 00000100:00100000:3.0:1789760787.337008:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507368320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.337057:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23416832, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.337066:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.337081:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.337086:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.337093:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.337108:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.361578:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760787.378121:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:3.0:1789760787.378140:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378154:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378164:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378173:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.378177:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378186:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378206:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.378213:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b988e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.378347:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507368320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41337us (42521us total) trans 4294973037 rc 0/0 00000100:00100000:3.0:1789760787.378358:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5747 00000100:00100000:3.0:1789760787.378379:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768554879354 time 768652295772 state 0x2 ref 0 00000001:00080000:1.0:1789760787.378504:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00000100:00100000:3.0:1789760787.381795:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760787.381901:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368448 00000100:00100000:2.0:1789760787.381910:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368448 00000100:00100000:2.0:1789760787.381963:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768554879354 time 768652295772 state 0x0 ref 1 00000100:00100000:2.0:1789760787.381977:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768652556188 time 768652295772 state 0x1 ref 1 00000100:00100000:2.0:1789760787.382007:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768652556188 time 768697310211 state 0x0 ref 1 00000100:00100000:2.0:1789760787.382020:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760787.382066:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23420928, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:2.0:1789760787.382076:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.382091:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760787.382095:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.382101:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.382115:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.418916:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00002000:00010000:0.0:1789760787.420825:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:0.0:1789760787.420846:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420859:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420869:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420877:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760787.420882:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420962:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420985:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.420995:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9895 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.421154:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39131us (39363us total) trans 4294973038 rc 0/0 00000100:00100000:0.0:1789760787.421166:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5748 00000100:00100000:0.0:1789760787.421185:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768652556188 time 768697310211 state 0x2 ref 0 00000100:00100000:0.0:1789760787.426133:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.426272:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368576 00000100:00100000:3.0:1789760787.426283:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368576 00000100:00100000:3.0:1789760787.426334:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768652556188 time 768697310211 state 0x0 ref 1 00000100:00100000:3.0:1789760787.426351:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768697570627 time 768697310211 state 0x1 ref 1 00000100:00100000:3.0:1789760787.426382:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768697570627 time 768741685429 state 0x0 ref 1 00000100:00100000:3.0:1789760787.426394:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.426452:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23425024, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.426462:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.426477:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.426481:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.426579:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.426595:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.462600:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00002000:00010000:3.0:1789760787.462739:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23425024 -> 23429120 00010000:00010000:3.0:1789760787.462755:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462767:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462775:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462782:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.462786:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462794:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462817:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.462825:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b989c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.463062:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36665us (36933us total) trans 4294973039 rc 0/0 00000100:00100000:3.0:1789760787.463072:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5749 00000100:00100000:3.0:1789760787.463082:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768697570627 time 768741685429 state 0x2 ref 0 00000100:00100000:2.0:1789760787.468803:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.468897:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368704 00000100:00100000:3.0:1789760787.468908:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368704 00000100:00100000:3.0:1789760787.468963:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768697570627 time 768741685429 state 0x0 ref 1 00000100:00100000:3.0:1789760787.468978:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768741945845 time 768741685429 state 0x1 ref 1 00000100:00100000:3.0:1789760787.469010:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768741945845 time 768784313292 state 0x0 ref 1 00000100:00100000:3.0:1789760787.469023:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507368704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.469070:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23429120, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.469080:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.469095:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.469100:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.469107:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.469121:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.507230:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00002000:00010000:3.0:1789760787.507424:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23429120 -> 23433216 00010000:00010000:3.0:1789760787.507439:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507460:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507469:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507477:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.507541:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507551:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507574:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.507582:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.507801:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507368704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38775us (39002us total) trans 4294973040 rc 0/0 00000100:00100000:3.0:1789760787.507813:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5750 00000100:00100000:3.0:1789760787.507822:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768741945845 time 768784313292 state 0x2 ref 0 00000100:00100000:0.0:1789760787.520354:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760787.520494:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368832 00000100:00100000:2.0:1789760787.520504:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368832 00000100:00100000:2.0:1789760787.520557:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768741945845 time 768784313292 state 0x0 ref 1 00000100:00100000:2.0:1789760787.520573:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768784573708 time 768784313292 state 0x1 ref 1 00000100:00100000:2.0:1789760787.520602:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768784573708 time 768835905140 state 0x0 ref 1 00000100:00100000:2.0:1789760787.520615:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760787.520663:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23433216, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:2.0:1789760787.520673:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520688:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760787.520730:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.520738:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520753:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.546803:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00002000:00010000:2.0:1789760787.546918:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:2.0:1789760787.546933:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.546946:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.546956:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.546962:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760787.546967:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.546976:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.546994:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.547001:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b98aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.547085:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507368832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26468us (26736us total) trans 4294973041 rc 0/0 00000100:00100000:2.0:1789760787.547095:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5751 00000100:00100000:2.0:1789760787.547104:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768784573708 time 768835905140 state 0x2 ref 0 00000100:00100000:1.0:1789760787.552891:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.553007:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507368960 00000100:00100000:3.0:1789760787.553017:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507368960 00000100:00100000:3.0:1789760787.553066:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768784573708 time 768835905140 state 0x0 ref 1 00000100:00100000:3.0:1789760787.553082:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768836165556 time 768835905140 state 0x1 ref 1 00000100:00100000:3.0:1789760787.553112:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768836165556 time 768868415749 state 0x0 ref 1 00000100:00100000:3.0:1789760787.553124:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507368960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.553175:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23437312, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.553183:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.553200:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.553205:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.553211:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.553225:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760787.586361:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:3.0:1789760787.586383:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586713:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586724:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586732:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.586737:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586746:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586765:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.586773:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.586905:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507368960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33778us (34019us total) trans 4294973042 rc 0/0 00000100:00100000:3.0:1789760787.586919:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5752 00000100:00100000:3.0:1789760787.586929:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768836165556 time 768868415749 state 0x2 ref 0 00000001:00080000:1.0:1789760787.587007:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00000100:00100000:0.0:1789760787.590393:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760787.590477:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369088 00000100:00100000:1.0:1789760787.590496:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369088 00000100:00100000:1.0:1789760787.590545:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768836165556 time 768868415749 state 0x0 ref 1 00000100:00100000:1.0:1789760787.590559:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768868676165 time 768868415749 state 0x1 ref 1 00000100:00100000:1.0:1789760787.590588:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768868676165 time 768905891359 state 0x0 ref 1 00000100:00100000:1.0:1789760787.590601:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760787.590645:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23441408, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:1.0:1789760787.590654:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.590669:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760787.590673:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.590678:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.590728:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.620673:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:3.0:1789760787.620846:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:3.0:1789760787.620862:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620874:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620882:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620889:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.620894:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620903:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620921:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.620929:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b98b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.621129:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30524us (30739us total) trans 4294973043 rc 0/0 00000100:00100000:3.0:1789760787.621139:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5753 00000100:00100000:3.0:1789760787.621148:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768868676165 time 768905891359 state 0x2 ref 0 00000100:00100000:2.0:1789760787.631183:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760787.631284:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369216 00000100:00100000:0.0:1789760787.631295:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369216 00000100:00100000:0.0:1789760787.631349:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768868676165 time 768905891359 state 0x0 ref 1 00000100:00100000:0.0:1789760787.631363:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768906151775 time 768905891359 state 0x1 ref 1 00000100:00100000:0.0:1789760787.631432:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768906151775 time 768946734856 state 0x0 ref 1 00000100:00100000:0.0:1789760787.631456:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760787.631515:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23445504, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:0.0:1789760787.631525:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.631550:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760787.631555:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.631561:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.631577:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.670464:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00002000:00010000:2.0:1789760787.670883:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23445504 -> 23449600 00010000:00010000:2.0:1789760787.671013:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671026:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671036:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671043:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760787.671048:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671056:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671075:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.671084:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b98bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.671215:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39768us (40037us total) trans 4294973044 rc 0/0 00000100:00100000:2.0:1789760787.671226:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5754 00000100:00100000:2.0:1789760787.671235:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768906151775 time 768946734856 state 0x2 ref 0 00000100:00100000:2.0:1789760787.675062:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.675163:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369344 00000100:00100000:3.0:1789760787.675172:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369344 00000100:00100000:3.0:1789760787.675225:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768906151775 time 768946734856 state 0x0 ref 1 00000100:00100000:3.0:1789760787.675241:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768946995272 time 768946734856 state 0x1 ref 1 00000100:00100000:3.0:1789760787.675275:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768946995272 time 768990578173 state 0x0 ref 1 00000100:00100000:3.0:1789760787.675288:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.675335:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23449600, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.675344:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.675360:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760787.675364:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.675370:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.675384:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.699320:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:3.0:1789760787.699422:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:3.0:1789760787.699438:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699460:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699469:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699475:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.699490:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699500:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699520:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.699527:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.699621:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24330us (24563us total) trans 4294973045 rc 0/0 00000100:00100000:3.0:1789760787.699634:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5755 00000100:00100000:3.0:1789760787.699644:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768946995272 time 768990578173 state 0x2 ref 0 00000100:00100000:0.0:1789760787.703388:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.703651:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369472 00000100:00100000:3.0:1789760787.703660:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369472 00000100:00100000:3.0:1789760787.703749:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768946995272 time 768990578173 state 0x0 ref 1 00000100:00100000:3.0:1789760787.703764:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768990838589 time 768990578173 state 0x1 ref 1 00000100:00100000:3.0:1789760787.703796:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768990838589 time 769019099194 state 0x0 ref 1 00000100:00100000:3.0:1789760787.703807:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.703850:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23453696, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.703859:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.703874:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760787.703878:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.703883:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.703909:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.731790:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:3.0:1789760787.731880:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:3.0:1789760787.731895:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.731909:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.731918:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.731925:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760787.731930:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.731938:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.732029:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.732038:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.732168:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28356us (28782us total) trans 4294973046 rc 0/0 00000100:00100000:3.0:1789760787.732180:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5756 00000100:00100000:3.0:1789760787.732190:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768990838589 time 769019099194 state 0x2 ref 0 00000100:00100000:2.0:1789760787.739864:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760787.740196:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369600 00000100:00100000:0.0:1789760787.740207:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369600 00000100:00100000:0.0:1789760787.740260:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 768990838589 time 769019099194 state 0x0 ref 1 00000100:00100000:0.0:1789760787.740274:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769019359610 time 769019099194 state 0x1 ref 1 00000100:00100000:0.0:1789760787.740307:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769019359610 time 769055610000 state 0x0 ref 1 00000100:00100000:0.0:1789760787.740320:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760787.740371:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23457792, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:0.0:1789760787.740380:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.740396:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760787.740400:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.740407:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.740422:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.810873:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:0.0:1789760787.811253:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23457792 -> 23461888 00010000:00010000:0.0:1789760787.811270:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811282:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811291:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811297:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760787.811301:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811309:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811330:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.811337:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.811428:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71106us (71568us total) trans 4294973047 rc 0/0 00000100:00100000:0.0:1789760787.811451:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5757 00000100:00100000:0.0:1789760787.811461:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769019359610 time 769055610000 state 0x2 ref 0 00000100:00100000:2.0:1789760787.817458:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.818217:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369728 00000100:00100000:3.0:1789760787.818228:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369728 00000100:00100000:3.0:1789760787.818283:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769019359610 time 769055610000 state 0x0 ref 1 00000100:00100000:3.0:1789760787.818298:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769055870416 time 769055610000 state 0x1 ref 1 00000100:00100000:3.0:1789760787.818327:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769055870416 time 769133630457 state 0x0 ref 1 00000100:00100000:3.0:1789760787.818338:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.818389:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23461888, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.818398:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.818411:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760787.818415:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.818422:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.818436:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.916432:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:1.0:1789760787.918791:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:1.0:1789760787.918810:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918823:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918833:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918840:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760787.918845:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918855:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918874:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.918883:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b98db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.919035:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 100693us (101591us total) trans 4294973048 rc 0/0 00000100:00100000:1.0:1789760787.919046:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5758 00000100:00100000:1.0:1789760787.919056:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769055870416 time 769133630457 state 0x2 ref 0 00000100:00100000:2.0:1789760787.925148:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760787.925208:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369856 00000100:00100000:3.0:1789760787.925217:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369856 00000100:00100000:3.0:1789760787.925271:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769055870416 time 769133630457 state 0x0 ref 1 00000100:00100000:3.0:1789760787.925286:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769133890873 time 769133630457 state 0x1 ref 1 00000100:00100000:3.0:1789760787.925318:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769133890873 time 769240621156 state 0x0 ref 1 00000100:00100000:3.0:1789760787.925331:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760787.925377:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23465984, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:3.0:1789760787.925386:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.925400:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760787.925405:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.925411:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.925425:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.981762:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00002000:00010000:1.0:1789760787.988675:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:1.0:1789760787.988892:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988910:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988919:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988926:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760787.988931:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988941:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988962:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988970:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b98e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.989074:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507369856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63738us (63930us total) trans 4294973049 rc 0/0 00000100:00100000:1.0:1789760787.989084:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5759 00000100:00100000:1.0:1789760787.989093:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769133890873 time 769240621156 state 0x2 ref 0 00000100:00100000:2.0:1789760787.994537:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760787.994655:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507369984 00000100:00100000:0.0:1789760787.994664:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507369984 00000100:00100000:0.0:1789760787.994747:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769133890873 time 769240621156 state 0x0 ref 1 00000100:00100000:0.0:1789760787.994762:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769240881572 time 769240621156 state 0x1 ref 1 00000100:00100000:0.0:1789760787.994782:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769240881572 time 769310085791 state 0x0 ref 1 00000100:00100000:0.0:1789760787.994796:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760787.994841:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23470080, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xb807 00010000:00010000:0.0:1789760787.994850:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.994864:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760787.994868:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.994874:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.994888:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.022271:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00002000:00010000:0.0:1789760788.022489:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23470080 -> 23474176 00002000:00010000:0.0:1789760788.022494:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:0.0:1789760788.022498:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:0.0:1789760788.022511:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022525:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022534:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022541:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.022545:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022554:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022573:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.022581:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.022672:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507369984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27875us (28141us total) trans 4294973050 rc 0/0 00000100:00100000:0.0:1789760788.022683:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5760 00000100:00100000:0.0:1789760788.022733:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769240881572 time 769310085791 state 0x2 ref 0 00000100:00100000:2.0:1789760788.026795:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.026918:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370112 00000100:00100000:1.0:1789760788.026928:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370112 00000100:00100000:1.0:1789760788.026980:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769240881572 time 769310085791 state 0x0 ref 1 00000100:00100000:1.0:1789760788.026994:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769310346207 time 769310085791 state 0x1 ref 1 00000100:00100000:1.0:1789760788.027015:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769310346207 time 769342318909 state 0x0 ref 1 00000100:00100000:1.0:1789760788.027027:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507370112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760788.027079:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23474176, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:1.0:1789760788.027089:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.027104:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760788.027108:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.027115:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.027130:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.071755:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:0.0:1789760788.071896:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:0.0:1789760788.071911:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071924:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071934:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071942:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.071947:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071958:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071977:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.071986:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b98f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.072077:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507370112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45046us (45285us total) trans 4294973051 rc 0/0 00000100:00100000:0.0:1789760788.072088:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5761 00000100:00100000:0.0:1789760788.072098:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769310346207 time 769342318909 state 0x2 ref 0 00000100:00100000:2.0:1789760788.075183:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760788.075484:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370240 00000100:00100000:3.0:1789760788.075494:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370240 00000100:00100000:3.0:1789760788.075545:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769310346207 time 769342318909 state 0x0 ref 1 00000100:00100000:3.0:1789760788.075561:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769342579325 time 769342318909 state 0x1 ref 1 00000100:00100000:3.0:1789760788.075589:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769342579325 time 769390892981 state 0x0 ref 1 00000100:00100000:3.0:1789760788.075602:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507370240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.075648:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23478272, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.075657:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.075678:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760788.075683:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.075689:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.075730:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.106089:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00002000:00010000:1.0:1789760788.106184:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:1.0:1789760788.106200:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106213:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106227:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106235:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760788.106240:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106250:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106270:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.106279:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b98f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.106408:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507370240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30803us (31232us total) trans 4294973052 rc 0/0 00000100:00100000:1.0:1789760788.106419:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5762 00000100:00100000:1.0:1789760788.106428:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769342579325 time 769390892981 state 0x2 ref 0 00000100:00100000:2.0:1789760788.110947:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.111040:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370368 00000100:00100000:0.0:1789760788.111050:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370368 00000100:00100000:0.0:1789760788.111105:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769342579325 time 769390892981 state 0x0 ref 1 00000100:00100000:0.0:1789760788.111120:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769391153397 time 769390892981 state 0x1 ref 1 00000100:00100000:0.0:1789760788.111180:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769391153397 time 769426482769 state 0x0 ref 1 00000100:00100000:0.0:1789760788.111194:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507370368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760788.111241:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23482368, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760788.111251:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.111267:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760788.111271:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.111278:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.111291:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.136247:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00002000:00010000:0.0:1789760788.136357:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:0.0:1789760788.136371:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136388:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136398:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136405:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.136410:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136420:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136449:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.136457:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b98fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.136590:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507370368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25393us (25705us total) trans 4294973053 rc 0/0 00000100:00100000:0.0:1789760788.136601:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5763 00000100:00100000:0.0:1789760788.136610:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769391153397 time 769426482769 state 0x2 ref 0 00000100:00100000:2.0:1789760788.145492:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.145608:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370496 00000100:00100000:0.0:1789760788.145617:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370496 00000100:00100000:0.0:1789760788.145669:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769391153397 time 769426482769 state 0x0 ref 1 00000100:00100000:0.0:1789760788.145683:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769426743185 time 769426482769 state 0x1 ref 1 00000100:00100000:2.0:1789760788.145824:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769426743185 time 769461124574 state 0x0 ref 1 00000100:00100000:2.0:1789760788.145837:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507370496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760788.145884:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23486464, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:2.0:1789760788.145893:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.145909:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760788.145914:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.145920:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.145939:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.166968:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00002000:00010000:3.0:1789760788.167065:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:3.0:1789760788.167081:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167099:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167176:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167183:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760788.167187:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167196:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167221:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.167228:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9905 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.167322:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507370496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21482us (21842us total) trans 4294973054 rc 0/0 00000100:00100000:3.0:1789760788.167333:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5764 00000100:00100000:3.0:1789760788.167342:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769426743185 time 769461124574 state 0x2 ref 0 00000100:00100000:2.0:1789760788.171169:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.171373:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370624 00000100:00100000:1.0:1789760788.171383:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370624 00000100:00100000:1.0:1789760788.171437:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769426743185 time 769461124574 state 0x0 ref 1 00000100:00100000:1.0:1789760788.171465:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769461384990 time 769461124574 state 0x1 ref 1 00000100:00100000:2.0:1789760788.172250:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769461384990 time 769487545922 state 0x0 ref 1 00000100:00100000:2.0:1789760788.172269:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507370624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760788.172319:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23490560, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:2.0:1789760788.172330:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.172345:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760788.172350:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.172357:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.172371:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.203168:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23490560 -> 23494656 00010000:00010000:1.0:1789760788.203189:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203202:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203211:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203218:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760788.203223:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203233:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203252:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.203259:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b990c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.203361:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507370624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31092us (32196us total) trans 4294973055 rc 0/0 00000100:00100000:1.0:1789760788.203371:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5765 00000100:00100000:1.0:1789760788.203381:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769461384990 time 769487545922 state 0x2 ref 0 00000001:00080000:3.0:1789760788.204013:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00000100:00100000:2.0:1789760788.207385:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.207618:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370752 00000100:00100000:0.0:1789760788.207628:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370752 00000100:00100000:0.0:1789760788.207679:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769461384990 time 769487545922 state 0x0 ref 1 00000100:00100000:0.0:1789760788.207867:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769487806338 time 769487545922 state 0x1 ref 1 00000100:00100000:0.0:1789760788.207905:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769487806338 time 769523207931 state 0x0 ref 1 00000100:00100000:0.0:1789760788.207919:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507370752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760788.207965:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23494656, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760788.207975:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.207991:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760788.207995:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.208002:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.208015:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.246824:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:1.0:1789760788.246844:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246857:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246866:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246873:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760788.246878:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246898:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246916:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.246923:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9913 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.247015:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507370752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39094us (39633us total) trans 4294973056 rc 0/0 00000100:00100000:1.0:1789760788.247030:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5766 00000100:00100000:1.0:1789760788.247041:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769487806338 time 769523207931 state 0x2 ref 0 00000001:00080000:3.0:1789760788.247178:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00000100:00100000:0.0:1789760788.252796:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760788.253109:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507370880 00000100:00100000:3.0:1789760788.253119:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507370880 00000100:00100000:3.0:1789760788.253172:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769487806338 time 769523207931 state 0x0 ref 1 00000100:00100000:3.0:1789760788.253188:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769523468347 time 769523207931 state 0x1 ref 1 00000100:00100000:3.0:1789760788.253219:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769523468347 time 769568521714 state 0x0 ref 1 00000100:00100000:3.0:1789760788.253232:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507370880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.253349:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23498752, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.253358:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.253378:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760788.253382:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.253389:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.253403:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.297172:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00002000:00010000:3.0:1789760788.297288:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:3.0:1789760788.297305:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297319:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297329:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297336:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760788.297341:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297350:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297371:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297379:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b991a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.297632:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23ebeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507370880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44397us (44841us total) trans 4294973057 rc 0/0 00000100:00100000:3.0:1789760788.297643:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5767 00000100:00100000:3.0:1789760788.297652:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769523468347 time 769568521714 state 0x2 ref 0 00000100:00100000:0.0:1789760788.302161:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.302241:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371008 00000100:00100000:1.0:1789760788.302251:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371008 00000100:00100000:1.0:1789760788.302304:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769523468347 time 769568521714 state 0x0 ref 1 00000100:00100000:1.0:1789760788.302318:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769568782130 time 769568521714 state 0x1 ref 1 00000100:00100000:1.0:1789760788.302348:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769568782130 time 769617650907 state 0x0 ref 1 00000100:00100000:1.0:1789760788.302363:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507371008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760788.302412:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23502848, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:1.0:1789760788.302421:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.302445:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760788.302450:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.302456:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.302546:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.341262:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:0.0:1789760788.342334:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23502848 -> 23506944 00010000:00010000:0.0:1789760788.342350:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342372:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342382:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342389:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.342394:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342404:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342424:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.342431:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9921 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.342540:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507371008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40177us (40385us total) trans 4294973058 rc 0/0 00000100:00100000:0.0:1789760788.342550:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5768 00000100:00100000:0.0:1789760788.342561:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769568782130 time 769617650907 state 0x2 ref 0 00000100:00100000:0.0:1789760788.345140:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.345284:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371136 00000100:00100000:1.0:1789760788.345294:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371136 00000100:00100000:1.0:1789760788.345350:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769568782130 time 769617650907 state 0x0 ref 1 00000100:00100000:1.0:1789760788.345366:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769617911323 time 769617650907 state 0x1 ref 1 00000100:00100000:1.0:1789760788.345397:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769617911323 time 769660699692 state 0x0 ref 1 00000100:00100000:1.0:1789760788.345409:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507371136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760788.345466:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23506944, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:1.0:1789760788.345482:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.345497:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760788.345502:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.345508:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.345522:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.380291:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760788.384446:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00002000:00010000:0.0:1789760788.384463:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23506944 -> 23511040 00010000:00010000:0.0:1789760788.384479:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384492:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384501:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384508:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.384514:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384523:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384543:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.384551:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9928 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.384653:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507371136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39240us (39516us total) trans 4294973059 rc 0/0 00000100:00100000:0.0:1789760788.384664:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5769 00000100:00100000:0.0:1789760788.384675:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769617911323 time 769660699692 state 0x2 ref 0 00000100:00100000:2.0:1789760788.392989:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760788.393137:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371264 00000100:00100000:3.0:1789760788.393147:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371264 00000100:00100000:3.0:1789760788.393201:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769617911323 time 769660699692 state 0x0 ref 1 00000100:00100000:3.0:1789760788.393215:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769660960108 time 769660699692 state 0x1 ref 1 00000100:00100000:3.0:1789760788.393249:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769660960108 time 769708551901 state 0x0 ref 1 00000100:00100000:3.0:1789760788.393261:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.393310:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23511040, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.393318:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.393334:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760788.393338:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.393345:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.393359:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.429685:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00002000:00010000:1.0:1789760788.430040:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:1.0:1789760788.430055:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430069:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430084:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430092:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760788.430097:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430104:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430123:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.430131:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b992f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.430222:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36957us (37238us total) trans 4294973060 rc 0/0 00000100:00100000:1.0:1789760788.430232:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5770 00000100:00100000:1.0:1789760788.430242:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769660960108 time 769708551901 state 0x2 ref 0 00000100:00100000:2.0:1789760788.435240:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.435327:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371392 00000100:00100000:0.0:1789760788.435336:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371392 00000100:00100000:0.0:1789760788.435753:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769660960108 time 769708551901 state 0x0 ref 1 00000100:00100000:0.0:1789760788.435768:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769708812317 time 769708551901 state 0x1 ref 1 00000100:00100000:0.0:1789760788.435795:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769708812317 time 769751099325 state 0x0 ref 1 00000100:00100000:0.0:1789760788.435807:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760788.435850:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23515136, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760788.435858:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.435874:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760788.435878:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.435883:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.435897:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.505601:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00002000:00010000:0.0:1789760788.506039:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:0.0:1789760788.506054:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506068:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506077:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506085:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.506089:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506098:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506117:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.506125:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9936 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.506331:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70520us (71094us total) trans 4294973061 rc 0/0 00000100:00100000:0.0:1789760788.506342:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5771 00000100:00100000:0.0:1789760788.506351:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769708812317 time 769751099325 state 0x2 ref 0 00000100:00100000:0.0:1789760788.509924:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.510187:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371520 00000100:00100000:1.0:1789760788.510198:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371520 00000100:00100000:1.0:1789760788.510255:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769708812317 time 769751099325 state 0x0 ref 1 00000100:00100000:1.0:1789760788.510269:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769751359741 time 769751099325 state 0x1 ref 1 00000100:00100000:1.0:1789760788.510302:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769751359741 time 769825605063 state 0x0 ref 1 00000100:00100000:1.0:1789760788.510315:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760788.510366:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23519232, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:1.0:1789760788.510374:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.510389:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760788.510394:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.510401:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.510418:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760788.581761:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:2.0:1789760788.581783:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581797:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581807:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581815:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760788.581820:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581830:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581853:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.581861:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007afb942b/0x9638aab0164b993d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.581957:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507371520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 71640us (72037us total) trans 4294973062 rc 0/0 00000100:00100000:2.0:1789760788.581968:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5772 00000100:00100000:2.0:1789760788.581977:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769751359741 time 769825605063 state 0x2 ref 0 00000001:00080000:3.0:1789760788.582049:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00000100:00100000:2.0:1789760788.585493:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.585620:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371648 00000100:00100000:1.0:1789760788.585630:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371648 00000100:00100000:1.0:1789760788.585684:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769751359741 time 769825605063 state 0x0 ref 1 00000100:00100000:1.0:1789760788.585730:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769825865479 time 769825605063 state 0x1 ref 1 00000100:00100000:1.0:1789760788.585762:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769825865479 time 769901065152 state 0x0 ref 1 00000100:00100000:1.0:1789760788.585775:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760788.585830:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23523328, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:1.0:1789760788.585839:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.585855:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760788.585860:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.585867:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.585882:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.618219:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:3.0:1789760788.618323:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:3.0:1789760788.618340:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618352:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618361:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618369:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760788.618374:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618382:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618402:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.618415:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9944 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.618568:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32789us (33089us total) trans 4294973063 rc 0/0 00000100:00100000:3.0:1789760788.618578:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5773 00000100:00100000:3.0:1789760788.618587:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769825865479 time 769901065152 state 0x2 ref 0 00000100:00100000:1.0:1789760788.622579:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.622730:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371776 00000100:00100000:0.0:1789760788.622740:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371776 00000100:00100000:0.0:1789760788.622792:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769825865479 time 769901065152 state 0x0 ref 1 00000100:00100000:0.0:1789760788.622807:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769901325568 time 769901065152 state 0x1 ref 1 00000100:00100000:0.0:1789760788.622833:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769901325568 time 769938136502 state 0x0 ref 1 00000100:00100000:0.0:1789760788.622846:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507371776:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760788.623032:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23527424, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760788.623042:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.623059:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760788.623064:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.623070:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.623085:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.670142:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:1.0:1789760788.670257:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:1.0:1789760788.670273:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670287:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670296:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670303:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760788.670309:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670317:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670337:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.670344:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b994b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.670446:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507371776:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47588us (47862us total) trans 4294973064 rc 0/0 00000100:00100000:1.0:1789760788.670456:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5774 00000100:00100000:1.0:1789760788.670466:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769901325568 time 769938136502 state 0x2 ref 0 00000100:00100000:0.0:1789760788.674072:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760788.674224:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507371904 00000100:00100000:2.0:1789760788.674233:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507371904 00000100:00100000:2.0:1789760788.674287:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769901325568 time 769938136502 state 0x0 ref 1 00000100:00100000:2.0:1789760788.674303:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769938396918 time 769938136502 state 0x1 ref 1 00000100:00100000:2.0:1789760788.674334:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769938396918 time 769989637864 state 0x0 ref 1 00000100:00100000:2.0:1789760788.674347:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371904:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760788.674405:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23531520, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:2.0:1789760788.674415:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.674431:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760788.674445:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.674452:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.674468:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.722377:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00002000:00010000:0.0:1789760788.722493:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:0.0:1789760788.722512:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722527:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722537:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722545:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.722550:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722560:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722585:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.722594:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9952 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.722871:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507371904:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48522us (48805us total) trans 4294973065 rc 0/0 00000100:00100000:0.0:1789760788.722882:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5775 00000100:00100000:0.0:1789760788.722891:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769938396918 time 769989637864 state 0x2 ref 0 00000100:00100000:0.0:1789760788.730089:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760788.730334:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372032 00000100:00100000:3.0:1789760788.730347:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372032 00000100:00100000:3.0:1789760788.730424:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769938396918 time 769989637864 state 0x0 ref 1 00000100:00100000:3.0:1789760788.730453:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769989898280 time 769989637864 state 0x1 ref 1 00000100:00100000:3.0:1789760788.730750:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769989898280 time 770046050853 state 0x0 ref 1 00000100:00100000:3.0:1789760788.730766:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372032:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.730839:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23535616, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.730850:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.730870:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760788.730875:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.730883:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.730905:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.800315:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00002000:00010000:3.0:1789760788.800519:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23535616 -> 23539712 00010000:00010000:3.0:1789760788.800540:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800556:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800566:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800574:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760788.800580:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800591:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800617:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.800625:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9959 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.800811:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372032:12345-192.168.203.41@tcp:4:dd.0 Request processed in 70040us (70728us total) trans 4294973066 rc 0/0 00000100:00100000:3.0:1789760788.800943:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5776 00000100:00100000:3.0:1789760788.800957:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769989898280 time 770046050853 state 0x2 ref 0 00000100:00100000:2.0:1789760788.809884:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760788.809972:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372160 00000100:00100000:1.0:1789760788.809983:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372160 00000100:00100000:1.0:1789760788.810049:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 769989898280 time 770046050853 state 0x0 ref 1 00000100:00100000:1.0:1789760788.810063:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770046311269 time 770046050853 state 0x1 ref 1 00000100:00100000:3.0:1789760788.810189:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770046311269 time 770125490325 state 0x0 ref 1 00000100:00100000:3.0:1789760788.810203:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372160:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.810269:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23539712, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.810280:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.810300:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760788.810305:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.810313:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.810330:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.893106:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:0.0:1789760788.893305:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:0.0:1789760788.893322:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893335:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893344:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893617:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.893622:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893632:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893652:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893667:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9960 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.893795:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372160:12345-192.168.203.41@tcp:4:dd.0 Request processed in 83584us (83922us total) trans 4294973067 rc 0/0 00000100:00100000:0.0:1789760788.893806:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5777 00000100:00100000:0.0:1789760788.893816:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770046311269 time 770125490325 state 0x2 ref 0 00000100:00100000:3.0:1789760788.902795:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760788.902881:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372288 00000100:00100000:0.0:1789760788.902891:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372288 00000100:00100000:0.0:1789760788.902945:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770046311269 time 770125490325 state 0x0 ref 1 00000100:00100000:0.0:1789760788.902962:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770125750741 time 770125490325 state 0x1 ref 1 00000100:00100000:0.0:1789760788.902995:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770125750741 time 770218298964 state 0x0 ref 1 00000100:00100000:0.0:1789760788.903009:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507372288:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760788.903060:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23543808, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760788.903069:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.903084:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760788.903089:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.903096:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.903111:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.941931:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00002000:00010000:0.0:1789760788.942015:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:0.0:1789760788.942031:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942045:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942056:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942063:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760788.942068:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942077:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942097:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942104:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9967 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.942197:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507372288:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39186us (39407us total) trans 4294973068 rc 0/0 00000100:00100000:0.0:1789760788.942207:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5778 00000100:00100000:0.0:1789760788.942217:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770125750741 time 770218298964 state 0x2 ref 0 00000100:00100000:0.0:1789760788.948949:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760788.949039:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372416 00000100:00100000:3.0:1789760788.949049:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372416 00000100:00100000:3.0:1789760788.949103:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770125750741 time 770218298964 state 0x0 ref 1 00000100:00100000:3.0:1789760788.949121:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770218559380 time 770218298964 state 0x1 ref 1 00000100:00100000:3.0:1789760788.949161:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770218559380 time 770264464250 state 0x0 ref 1 00000100:00100000:3.0:1789760788.949175:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760788.949225:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23547904, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:3.0:1789760788.949234:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.949250:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760788.949254:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.949260:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.949273:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.006009:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:0.0:1789760789.006029:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006047:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006057:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006064:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.006069:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006079:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006100:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.006107:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b996e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.006202:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507372416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57025us (57259us total) trans 4294973069 rc 0/0 00000100:00100000:0.0:1789760789.006213:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5779 00000100:00100000:0.0:1789760789.006222:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770218559380 time 770264464250 state 0x2 ref 0 00000001:00080000:1.0:1789760789.006373:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00000100:00100000:2.0:1789760789.013380:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.013587:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372544 00000100:00100000:0.0:1789760789.013597:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372544 00000100:00100000:0.0:1789760789.013671:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770218559380 time 770264464250 state 0x0 ref 1 00000100:00100000:0.0:1789760789.013686:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770264724666 time 770264464250 state 0x1 ref 1 00000100:00100000:0.0:1789760789.013743:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770264724666 time 770329046462 state 0x0 ref 1 00000100:00100000:0.0:1789760789.013757:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507372544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.013808:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23552000, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xb807 00010000:00010000:0.0:1789760789.013816:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013832:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760789.013836:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.013842:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013856:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760789.014419:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760789.042375:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23552000 -> 23556096 00002000:00010000:0.0:1789760789.042384:0:8092:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:0.0:1789760789.042386:0:8092:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00010000:00010000:0.0:1789760789.042399:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042412:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042421:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042428:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.042442:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042452:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042659:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.042669:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9975 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.042799:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507372544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29040us (29423us total) trans 4294973070 rc 0/0 00000100:00100000:0.0:1789760789.042810:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5780 00000100:00100000:0.0:1789760789.042819:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770264724666 time 770329046462 state 0x2 ref 0 00000001:00080000:1.0:1789760789.044411:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00000100:00100000:0.0:1789760789.052643:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760789.052722:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372672 00000100:00100000:2.0:1789760789.052731:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372672 00000100:00100000:2.0:1789760789.052781:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770264724666 time 770329046462 state 0x0 ref 1 00000100:00100000:2.0:1789760789.052795:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770329306878 time 770329046462 state 0x1 ref 1 00000100:00100000:2.0:1789760789.052855:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770329306878 time 770368158412 state 0x0 ref 1 00000100:00100000:2.0:1789760789.052867:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507372672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760789.052909:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23556096, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:2.0:1789760789.052918:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052933:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760789.052938:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.052943:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052956:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.117634:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00002000:00010000:3.0:1789760789.117775:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:3.0:1789760789.117794:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117806:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117814:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117821:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.117826:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117835:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117855:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.117864:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b997c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.118015:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507372672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65143us (65373us total) trans 4294973071 rc 0/0 00000100:00100000:3.0:1789760789.118025:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5781 00000100:00100000:3.0:1789760789.118034:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770329306878 time 770368158412 state 0x2 ref 0 00000100:00100000:0.0:1789760789.122797:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.126130:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372800 00000100:00100000:3.0:1789760789.126147:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372800 00000100:00100000:3.0:1789760789.126219:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770329306878 time 770368158412 state 0x0 ref 1 00000100:00100000:3.0:1789760789.126238:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770368418828 time 770368158412 state 0x1 ref 1 00000100:00100000:3.0:1789760789.126270:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770368418828 time 770441574246 state 0x0 ref 1 00000100:00100000:3.0:1789760789.126286:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507372800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.126329:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23560192, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.126337:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.126351:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.126356:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.126362:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.126376:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.160998:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00002000:00010000:3.0:1789760789.161100:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:3.0:1789760789.161116:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161130:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161140:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161147:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.161151:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161160:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161179:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.161187:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9983 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.161291:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507372800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35002us (38500us total) trans 4294973072 rc 0/0 00000100:00100000:3.0:1789760789.161302:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5782 00000100:00100000:3.0:1789760789.161311:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770368418828 time 770441574246 state 0x2 ref 0 00000100:00100000:0.0:1789760789.164838:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760789.164953:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507372928 00000100:00100000:2.0:1789760789.164963:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507372928 00000100:00100000:2.0:1789760789.165014:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770368418828 time 770441574246 state 0x0 ref 1 00000100:00100000:2.0:1789760789.165028:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770441834662 time 770441574246 state 0x1 ref 1 00000100:00100000:2.0:1789760789.165090:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770441834662 time 770480393265 state 0x0 ref 1 00000100:00100000:2.0:1789760789.165105:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414d4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507372928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760789.165189:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23564288, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:2.0:1789760789.165209:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.165224:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760789.165228:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.165233:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.165245:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.203804:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00002000:00010000:3.0:1789760789.203989:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:3.0:1789760789.204005:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204019:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204029:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204038:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.204044:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204053:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204072:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.204079:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b998a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.204172:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414d4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507372928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39064us (39338us total) trans 4294973073 rc 0/0 00000100:00100000:3.0:1789760789.204182:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5783 00000100:00100000:3.0:1789760789.204191:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770441834662 time 770480393265 state 0x2 ref 0 00000100:00100000:1.0:1789760789.210867:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211608:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211648:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211676:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211721:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211761:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.211800:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373312 00000100:00100000:1.0:1789760789.211810:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373312 00000100:00100000:1.0:1789760789.211867:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373568 00000100:00100000:1.0:1789760789.211870:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373568 00000100:00100000:1.0:1789760789.211900:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373184 00000100:00100000:1.0:1789760789.211903:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373184 00000100:00100000:1.0:1789760789.211915:0:6442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373696 00000100:00100000:1.0:1789760789.211918:0:6442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373696 00000100:00100000:1.0:1789760789.211932:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 978 00000100:00100000:1.0:1789760789.211942:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507373312:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760789.212009:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507373312:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 68us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.212017:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 978 00000100:00100000:1.0:1789760789.212032:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 979 00000100:00100000:1.0:1789760789.212039:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507373568:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760789.212063:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2419:x1876699507373568:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.212070:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 979 00000100:00100000:1.0:1789760789.212080:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 980 00000100:00100000:1.0:1789760789.212086:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507373184:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760789.212109:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:7fdb621d-3376-4597-a0ab-d89800273221+8:2418:x1876699507373184:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 23us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.212115:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 980 00000100:00100000:1.0:1789760789.212126:0:6442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 981 00000100:00100000:1.0:1789760789.212132:0:6442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507373696:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760789.212154:0:6442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2418:x1876699507373696:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 22us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.212159:0:6442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 981 00000100:00100000:1.0:1789760789.212176:0:8085:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373440 00000100:00100000:1.0:1789760789.212179:0:8085:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373440 00000100:00100000:1.0:1789760789.212208:0:8085:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 341 00000100:00100000:1.0:1789760789.212215:0:8085:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507373440:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:1.0:1789760789.212237:0:8085:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507373440:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 24us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.212243:0:8085:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 341 00000100:00080000:1.0:1789760789.212275:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760789.212280:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760789.212575:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373056 00000100:00100000:2.0:1789760789.212580:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373056 00000100:00100000:2.0:1789760789.212618:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 486 00000100:00100000:2.0:1789760789.212625:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507373056:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:2.0:1789760789.212654:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2419:x1876699507373056:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 30us (1798us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.212661:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 486 00000100:00100000:0.0:1789760789.214421:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.214600:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373824 00000100:00100000:3.0:1789760789.214609:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373824 00000100:00100000:3.0:1789760789.214653:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770441834662 time 770480393265 state 0x0 ref 1 00000100:00100000:3.0:1789760789.214668:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770480653681 time 770480393265 state 0x1 ref 1 00000100:00100000:3.0:1789760789.214724:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770480653681 time 770530027428 state 0x0 ref 1 00000100:00100000:3.0:1789760789.214736:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507373824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.215390:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23568384, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.215402:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.215417:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.215421:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.215427:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.215450:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.255966:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00002000:00010000:3.0:1789760789.256049:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:3.0:1789760789.256065:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256079:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256089:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256097:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.256103:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256112:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256137:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.256145:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9991 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.256331:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507373824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41590us (41911us total) trans 4294973074 rc 0/0 00000100:00100000:3.0:1789760789.256342:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5784 00000100:00100000:3.0:1789760789.256351:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770480653681 time 770530027428 state 0x2 ref 0 00000100:00100000:0.0:1789760789.262650:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.262965:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507373952 00000100:00100000:3.0:1789760789.262976:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507373952 00000100:00100000:2.0:1789760789.263117:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770480653681 time 770530027428 state 0x0 ref 1 00000100:00100000:2.0:1789760789.263134:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770530287844 time 770530027428 state 0x1 ref 1 00000100:00100000:2.0:1789760789.263200:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770530287844 time 770578502469 state 0x0 ref 1 00000100:00100000:2.0:1789760789.263213:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507373952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760789.263271:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23572480, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:2.0:1789760789.263280:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.263295:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760789.263300:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.263306:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.263319:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.297569:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00002000:00010000:3.0:1789760789.297714:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23572480 -> 23576576 00010000:00010000:3.0:1789760789.297728:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297737:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297919:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297928:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.297933:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297953:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297975:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.297983:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9998 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.298079:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507373952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34863us (35433us total) trans 4294973075 rc 0/0 00000100:00100000:3.0:1789760789.298089:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5785 00000100:00100000:3.0:1789760789.298098:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770530287844 time 770578502469 state 0x2 ref 0 00000100:00100000:1.0:1789760789.303684:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.304267:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374080 00000100:00100000:3.0:1789760789.304277:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374080 00000100:00100000:3.0:1789760789.304332:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770530287844 time 770578502469 state 0x0 ref 1 00000100:00100000:3.0:1789760789.304359:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770578762885 time 770578502469 state 0x1 ref 1 00000100:00100000:3.0:1789760789.304390:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770578762885 time 770619693194 state 0x0 ref 1 00000100:00100000:3.0:1789760789.304402:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.304458:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23576576, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.304468:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.304484:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.304488:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.304494:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.304508:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.335991:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00002000:00010000:0.0:1789760789.336048:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:0.0:1789760789.336062:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336075:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336083:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336090:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.336095:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336103:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336121:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336128:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b999f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.336225:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31820us (32546us total) trans 4294973076 rc 0/0 00000100:00100000:0.0:1789760789.336235:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5786 00000100:00100000:0.0:1789760789.336244:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770578762885 time 770619693194 state 0x2 ref 0 00000100:00100000:2.0:1789760789.340036:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.340084:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374208 00000100:00100000:3.0:1789760789.340094:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374208 00000100:00100000:3.0:1789760789.340146:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770578762885 time 770619693194 state 0x0 ref 1 00000100:00100000:3.0:1789760789.340160:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770619953610 time 770619693194 state 0x1 ref 1 00000100:00100000:3.0:1789760789.340189:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770619953610 time 770655492381 state 0x0 ref 1 00000100:00100000:3.0:1789760789.340206:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507374208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.340251:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23580672, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.340258:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.340270:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.340273:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.340277:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.340291:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.368619:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00002000:00010000:3.0:1789760789.368763:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:3.0:1789760789.368777:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368790:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368799:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368820:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.368825:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368833:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368851:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.368859:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.369256:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507374208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29049us (29219us total) trans 4294973077 rc 0/0 00000100:00100000:2.0:1789760789.369268:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5787 00000100:00100000:2.0:1789760789.369278:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770619953610 time 770655492381 state 0x2 ref 0 00000100:00100000:2.0:1789760789.379589:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.379674:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374336 00000100:00100000:3.0:1789760789.379684:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374336 00000100:00100000:3.0:1789760789.379779:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770619953610 time 770655492381 state 0x0 ref 1 00000100:00100000:3.0:1789760789.379795:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770655752797 time 770655492381 state 0x1 ref 1 00000100:00100000:3.0:1789760789.379830:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770655752797 time 770695133485 state 0x0 ref 1 00000100:00100000:3.0:1789760789.379843:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760789.380046:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23584768, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:2.0:1789760789.380057:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.380073:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760789.380078:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.380084:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.380098:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.404317:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760789.452635:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:3.0:1789760789.453017:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:3.0:1789760789.453032:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453044:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453173:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453180:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.453185:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453194:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453220:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.453228:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b99ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.457540:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 77686us (77948us total) trans 4294973078 rc 0/0 00000100:00100000:3.0:1789760789.457555:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5788 00000100:00100000:3.0:1789760789.457565:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770655752797 time 770695133485 state 0x2 ref 0 00000100:00100000:1.0:1789760789.464894:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.465341:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374464 00000100:00100000:0.0:1789760789.465352:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374464 00000100:00100000:0.0:1789760789.465406:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770655752797 time 770695133485 state 0x0 ref 1 00000100:00100000:0.0:1789760789.465421:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770695393901 time 770695133485 state 0x1 ref 1 00000100:00100000:0.0:1789760789.465472:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770695393901 time 770780767345 state 0x0 ref 1 00000100:00100000:0.0:1789760789.465486:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.465531:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23588864, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:0.0:1789760789.465540:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.465554:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760789.465559:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.465566:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.465586:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.509208:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00002000:00010000:0.0:1789760789.509984:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:0.0:1789760789.509998:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510010:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510018:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510026:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.510031:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510040:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510061:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.510068:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.510171:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507374464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44684us (45292us total) trans 4294973079 rc 0/0 00000100:00100000:0.0:1789760789.510182:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5789 00000100:00100000:0.0:1789760789.510192:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770695393901 time 770780767345 state 0x2 ref 0 00000100:00100000:2.0:1789760789.517260:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.517351:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374592 00000100:00100000:0.0:1789760789.517360:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374592 00000100:00100000:0.0:1789760789.517416:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770695393901 time 770780767345 state 0x0 ref 1 00000100:00100000:0.0:1789760789.517442:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770781027761 time 770780767345 state 0x1 ref 1 00000100:00100000:0.0:1789760789.517519:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770781027761 time 770832820870 state 0x0 ref 1 00000100:00100000:0.0:1789760789.517532:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507374592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.517582:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23592960, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:0.0:1789760789.517592:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.517609:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760789.517614:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.517620:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.517635:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.553410:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00002000:00010000:3.0:1789760789.553605:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:3.0:1789760789.553620:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553632:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553640:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553648:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.553653:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553662:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553683:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.553814:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b99bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.553909:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507374592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36373us (36654us total) trans 4294973080 rc 0/0 00000100:00100000:3.0:1789760789.553920:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5790 00000100:00100000:3.0:1789760789.553930:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770781027761 time 770832820870 state 0x2 ref 0 00000100:00100000:2.0:1789760789.564914:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.565095:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374720 00000100:00100000:3.0:1789760789.565106:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374720 00000100:00100000:3.0:1789760789.565641:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770781027761 time 770832820870 state 0x0 ref 1 00000100:00100000:3.0:1789760789.565660:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770833081286 time 770832820870 state 0x1 ref 1 00000100:00100000:3.0:1789760789.565827:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770833081286 time 770881128420 state 0x0 ref 1 00000100:00100000:3.0:1789760789.565842:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507374720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.565904:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23597056, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.565913:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.565931:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.565936:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.565955:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.565970:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.619597:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:3.0:1789760789.619618:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619630:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619639:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619646:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.619651:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619658:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619679:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.619687:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.619809:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507374720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53967us (54899us total) trans 4294973081 rc 0/0 00000100:00100000:3.0:1789760789.619819:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5791 00000100:00100000:3.0:1789760789.619828:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770833081286 time 770881128420 state 0x2 ref 0 00000001:00080000:3.0:1789760789.619869:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00000100:00100000:1.0:1789760789.624228:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.624483:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374848 00000100:00100000:0.0:1789760789.624493:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374848 00000100:00100000:0.0:1789760789.624547:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770833081286 time 770881128420 state 0x0 ref 1 00000100:00100000:0.0:1789760789.624563:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770881388836 time 770881128420 state 0x1 ref 1 00000100:00100000:0.0:1789760789.624718:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770881388836 time 770940019104 state 0x0 ref 1 00000100:00100000:0.0:1789760789.624733:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507374848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.624782:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23601152, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:0.0:1789760789.624791:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.624807:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760789.624812:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.624818:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.624830:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.663313:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00002000:00010000:3.0:1789760789.663516:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23601152 -> 23605248 00010000:00010000:3.0:1789760789.663530:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663543:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663553:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663561:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.663566:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663575:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663602:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663610:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b99c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.663729:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507374848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38995us (39507us total) trans 4294973082 rc 0/0 00000100:00100000:3.0:1789760789.663753:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5792 00000100:00100000:3.0:1789760789.663762:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770881388836 time 770940019104 state 0x2 ref 0 00000100:00100000:1.0:1789760789.672222:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.672378:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507374976 00000100:00100000:3.0:1789760789.672391:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507374976 00000100:00100000:3.0:1789760789.672457:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770881388836 time 770940019104 state 0x0 ref 1 00000100:00100000:3.0:1789760789.673062:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770940279520 time 770940019104 state 0x1 ref 1 00000100:00100000:3.0:1789760789.673099:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770940279520 time 770988401510 state 0x0 ref 1 00000100:00100000:3.0:1789760789.673113:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507374976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.673161:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23605248, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.673170:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.676762:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760789.676774:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.676784:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.676807:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.724803:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538295040:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760789.725149:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.725205:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538295552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.725339:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234676d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538295424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.725374:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.725396:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538295936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.725726:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.725753:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538296448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.725778:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.725992:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:2.0:1789760789.726004:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234677b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538295296:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.726035:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.726059:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538295808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.726111:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.726131:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538296320:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.726154:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.726171:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538296832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.726191:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.726321:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295040 00000100:00100000:0.0:1789760789.726328:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295040 00000100:00100000:0.0:1789760789.726380:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789760789.726390:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538295040:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760789.726587:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.726609:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538296064:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.726631:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.726707:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538296576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.726731:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.726753:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1789760789.726759:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538295168:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760789.726779:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789760789.727011:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760789.727053:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538295040:12345-0@lo:13:osp-pre-1-1.0 Request processed in 665us (1904us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.727061:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:1.0:1789760789.727225:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538295680:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.727252:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.727280:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538296192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.727302:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.727318:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538296704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.727336:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760789.727372:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538295040:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1789760789.727406:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760789.727479:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295424 00000100:00100000:3.0:1789760789.727483:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295424 00000100:00100000:3.0:1789760789.727506:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295936 00000100:00100000:3.0:1789760789.727509:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295936 00000100:00100000:3.0:1789760789.727525:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296448 00000100:00100000:3.0:1789760789.727527:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296448 00000100:00100000:3.0:1789760789.727538:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296064 00000100:00100000:3.0:1789760789.727541:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296064 00000100:00100000:3.0:1789760789.727551:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296576 00000100:00100000:3.0:1789760789.727553:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296576 00000100:00100000:3.0:1789760789.727564:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295680 00000100:00100000:3.0:1789760789.727566:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295680 00000100:00100000:3.0:1789760789.727579:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 982 00000100:00100000:3.0:1789760789.727585:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538295424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.727633:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3464:x1876699538295424:12345-0@lo:400:kworker.0 Request processed in 48us (2259us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.727640:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 982 00000100:00100000:3.0:1789760789.727654:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 983 00000100:00100000:3.0:1789760789.727660:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538295936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.727689:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3464:x1876699538295936:12345-0@lo:400:kworker.0 Request processed in 30us (1964us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.727714:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 983 00000100:00100000:3.0:1789760789.727727:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 984 00000100:00100000:3.0:1789760789.727733:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538296448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.727759:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3464:x1876699538296448:12345-0@lo:400:kworker.0 Request processed in 27us (1981us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.727765:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 984 00000100:00100000:3.0:1789760789.727775:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 985 00000100:00100000:3.0:1789760789.727780:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538296064:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.727822:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3461:x1876699538296064:12345-0@lo:400:kworker.0 Request processed in 42us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.727828:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 985 00000100:00100000:3.0:1789760789.727837:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 986 00000100:00100000:3.0:1789760789.727842:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538296576:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.727866:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3461:x1876699538296576:12345-0@lo:400:kworker.0 Request processed in 24us (1135us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.727881:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 986 00000100:00100000:3.0:1789760789.728673:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 987 00000100:00100000:3.0:1789760789.728681:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538295680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.728739:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3461:x1876699538295680:12345-0@lo:400:kworker.0 Request processed in 61us (1487us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.728746:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 987 00000100:00100000:3.0:1789760789.730660:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295808 00000100:00100000:3.0:1789760789.730672:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295808 00000100:00100000:3.0:1789760789.730733:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295552 00000100:00100000:3.0:1789760789.730736:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295552 00000100:00100000:3.0:1789760789.730753:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760789.730762:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538295808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.730848:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538295808:12345-0@lo:400:kworker.0 Request processed in 86us (4735us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.730854:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760789.730869:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760789.731066:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538295552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.731104:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538295552:12345-0@lo:400:kworker.0 Request processed in 221us (4518us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.731110:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760789.731133:0:6435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295296 00000100:00100000:3.0:1789760789.731137:0:6435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295296 00000100:00100000:3.0:1789760789.731169:0:6435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789760789.731176:0:6435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c223a71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538295296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.731206:0:6435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c223a71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3464:x1876699538295296:12345-0@lo:400:kworker.0 Request processed in 30us (5170us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.731212:0:6435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:2.0:1789760789.731585:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296320 00000100:00100000:2.0:1789760789.731592:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296320 00000100:00100000:2.0:1789760789.731638:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296832 00000100:00100000:2.0:1789760789.731642:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296832 00000100:00100000:2.0:1789760789.731661:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296192 00000100:00100000:2.0:1789760789.731664:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296192 00000100:00100000:2.0:1789760789.731676:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296704 00000100:00100000:2.0:1789760789.731679:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296704 00000100:00100000:2.0:1789760789.731706:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789760789.731714:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538296320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.731753:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538296320:12345-0@lo:400:kworker.0 Request processed in 40us (5597us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.731759:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789760789.731774:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:2.0:1789760789.731780:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538296832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.731819:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538296832:12345-0@lo:400:kworker.0 Request processed in 39us (5626us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.731825:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:2.0:1789760789.731837:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1789760789.731844:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538296192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.732077:0:8086:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1789760789.732086:0:8086:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538296704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.732120:0:8086:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538296704:12345-0@lo:400:kworker.0 Request processed in 36us (4783us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.732126:0:8086:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1789760789.732171:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3461:x1876699538296192:12345-0@lo:400:kworker.0 Request processed in 30us (4569us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.732178:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:0.0:1789760789.732271:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538295168 00000100:00100000:0.0:1789760789.732276:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538295168 00000100:00100000:0.0:1789760789.732298:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760789.732304:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538295168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760789.732317:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760789.732349:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538295168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (5569us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.732355:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00080000:1.0:1789760789.734118:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760789.734126:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760789.734129:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760789.734132:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760789.734181:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538295168:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760789.734197:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3461:1876699538295680:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734208:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538296192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734286:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538296704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734298:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538295552:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734310:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3461:1876699538296064:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734321:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3461:1876699538296576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734391:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234677b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a12369fc-e31f-4eec-83dc-16c04178fd0b:3464:1876699538295296:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734406:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538295808:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734541:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538296320:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734554:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538296832:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734567:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234676d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3464:1876699538295424:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734580:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3464:1876699538295936:0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.734594:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23b459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3464:1876699538296448:0@lo:400:kworker.0 00000001:00080000:1.0:1789760789.736384:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00002000:00010000:3.0:1789760789.736660:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:3.0:1789760789.736675:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736688:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736727:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736735:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.736739:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736749:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736770:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.736778:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.737730:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507374976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64576us (65473us total) trans 4294973083 rc 0/0 00000100:00100000:3.0:1789760789.737745:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5793 00000100:00100000:3.0:1789760789.737755:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770940279520 time 770988401510 state 0x2 ref 0 00000100:00100000:0.0:1789760789.742094:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.742160:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375104 00000100:00100000:3.0:1789760789.742170:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375104 00000100:00100000:3.0:1789760789.742287:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770940279520 time 770988401510 state 0x0 ref 1 00000100:00100000:3.0:1789760789.742304:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770988661926 time 770988401510 state 0x1 ref 1 00000100:00100000:3.0:1789760789.742333:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770988661926 time 771057635863 state 0x0 ref 1 00000100:00100000:3.0:1789760789.742346:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507375104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.742395:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23609344, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.742404:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.742419:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760789.742424:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.742441:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.742456:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.764346:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:0.0:1789760789.764366:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764379:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764388:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764397:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.764401:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764417:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764447:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.764455:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.764557:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507375104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22209us (22465us total) trans 4294973084 rc 0/0 00000100:00100000:0.0:1789760789.764568:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5794 00000100:00100000:0.0:1789760789.764577:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770988661926 time 771057635863 state 0x2 ref 0 00000001:00080000:2.0:1789760789.764723:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00000100:00100000:1.0:1789760789.768219:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.768274:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375232 00000100:00100000:3.0:1789760789.768284:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375232 00000100:00100000:3.0:1789760789.768338:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 770988661926 time 771057635863 state 0x0 ref 1 00000100:00100000:3.0:1789760789.768353:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771057896279 time 771057635863 state 0x1 ref 1 00000100:00100000:3.0:1789760789.768440:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771057896279 time 771083731267 state 0x0 ref 1 00000100:00100000:3.0:1789760789.768453:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507375232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.768554:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23613440, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.768565:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.768683:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760789.768687:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.768819:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.768835:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.801028:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00002000:00010000:3.0:1789760789.801136:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23613440 -> 23617536 00010000:00010000:3.0:1789760789.801152:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801165:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801174:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801182:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760789.801186:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801202:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801227:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.801235:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.801370:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507375232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32908us (33149us total) trans 4294973085 rc 0/0 00000100:00100000:3.0:1789760789.801382:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5795 00000100:00100000:3.0:1789760789.801392:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771057896279 time 771083731267 state 0x2 ref 0 00000100:00100000:3.0:1789760789.804936:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.805162:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375360 00000100:00100000:0.0:1789760789.805173:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375360 00000100:00100000:0.0:1789760789.805227:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771057896279 time 771083731267 state 0x0 ref 1 00000100:00100000:0.0:1789760789.805241:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771083991683 time 771083731267 state 0x1 ref 1 00000100:00100000:0.0:1789760789.805319:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771083991683 time 771120621369 state 0x0 ref 1 00000100:00100000:0.0:1789760789.805332:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507375360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.805384:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23617536, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:0.0:1789760789.805395:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.805411:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760789.805415:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.805422:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.805447:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.829666:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:0.0:1789760789.830268:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:0.0:1789760789.830283:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830295:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830305:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830312:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760789.830317:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830326:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830352:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.830360:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b99e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.830458:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507375360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25122us (25526us total) trans 4294973086 rc 0/0 00000100:00100000:0.0:1789760789.830477:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5796 00000100:00100000:0.0:1789760789.830487:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771083991683 time 771120621369 state 0x2 ref 0 00000100:00100000:0.0:1789760789.836275:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.836383:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375488 00000100:00100000:3.0:1789760789.836393:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375488 00000100:00100000:3.0:1789760789.836455:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771083991683 time 771120621369 state 0x0 ref 1 00000100:00100000:3.0:1789760789.836479:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771120881785 time 771120621369 state 0x1 ref 1 00000100:00100000:3.0:1789760789.836509:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771120881785 time 771151812403 state 0x0 ref 1 00000100:00100000:3.0:1789760789.836522:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507375488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.836567:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23621632, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.836575:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.836596:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760789.836601:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.836607:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.836621:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.859200:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:1.0:1789760789.859351:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:1.0:1789760789.859365:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859378:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859388:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859395:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760789.859400:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859408:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859438:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.859446:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b99ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.859981:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b703100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507375488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23455us (23712us total) trans 4294973087 rc 0/0 00000100:00100000:1.0:1789760789.859992:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5797 00000100:00100000:1.0:1789760789.860001:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771120881785 time 771151812403 state 0x2 ref 0 00000100:00100000:1.0:1789760789.863150:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760789.863272:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375616 00000100:00100000:3.0:1789760789.863282:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375616 00000100:00100000:3.0:1789760789.863335:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771120881785 time 771151812403 state 0x0 ref 1 00000100:00100000:3.0:1789760789.863350:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771152072819 time 771151812403 state 0x1 ref 1 00000100:00100000:3.0:1789760789.863381:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771152072819 time 771178684238 state 0x0 ref 1 00000100:00100000:3.0:1789760789.863394:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507375616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760789.863455:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23625728, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:3.0:1789760789.863464:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.863575:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760789.863580:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.863587:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.863606:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.890292:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:1.0:1789760789.890404:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23625728 -> 23629824 00010000:00010000:1.0:1789760789.890420:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890443:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890452:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890460:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760789.890534:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890549:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890569:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.890576:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b99f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.890740:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507375616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27343us (27593us total) trans 4294973088 rc 0/0 00000100:00100000:1.0:1789760789.890750:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5798 00000100:00100000:1.0:1789760789.890760:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771152072819 time 771178684238 state 0x2 ref 0 00000100:00100000:3.0:1789760789.894464:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760789.895098:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375744 00000100:00100000:0.0:1789760789.895109:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375744 00000100:00100000:0.0:1789760789.895167:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771152072819 time 771178684238 state 0x0 ref 1 00000100:00100000:0.0:1789760789.895181:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771178944654 time 771178684238 state 0x1 ref 1 00000100:00100000:0.0:1789760789.895250:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771178944654 time 771210552808 state 0x0 ref 1 00000100:00100000:0.0:1789760789.895264:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507375744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760789.895314:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23629824, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:0.0:1789760789.895325:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.895344:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760789.895348:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.895354:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.895367:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.927282:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:1.0:1789760789.927301:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927314:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927323:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927331:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760789.927336:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927345:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927365:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.927372:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b99fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.927479:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507375744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32213us (33020us total) trans 4294973089 rc 0/0 00000100:00100000:1.0:1789760789.927489:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5799 00000100:00100000:1.0:1789760789.927499:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771178944654 time 771210552808 state 0x2 ref 0 00000001:00080000:2.0:1789760789.927623:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00000100:00100000:0.0:1789760789.931000:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.934586:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507375872 00000100:00100000:1.0:1789760789.934600:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507375872 00000100:00100000:1.0:1789760789.934662:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771178944654 time 771210552808 state 0x0 ref 1 00000100:00100000:1.0:1789760789.934680:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771210813224 time 771210552808 state 0x1 ref 1 00000100:00100000:1.0:1789760789.934728:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771210813224 time 771250031086 state 0x0 ref 1 00000100:00100000:1.0:1789760789.934740:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507375872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760789.934791:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23633920, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:1.0:1789760789.934800:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.934815:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760789.934820:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.934826:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.934841:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.971557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:1.0:1789760789.971625:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:1.0:1789760789.971641:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971653:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971661:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971667:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760789.971671:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971679:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971733:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.971740:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.971830:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507375872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37088us (40836us total) trans 4294973090 rc 0/0 00000100:00100000:1.0:1789760789.971841:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5800 00000100:00100000:1.0:1789760789.971849:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771210813224 time 771250031086 state 0x2 ref 0 00000100:00100000:0.0:1789760789.977768:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760789.977968:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376000 00000100:00100000:1.0:1789760789.977978:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376000 00000100:00100000:2.0:1789760789.978262:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771210813224 time 771250031086 state 0x0 ref 1 00000100:00100000:2.0:1789760789.978281:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771250291502 time 771250031086 state 0x1 ref 1 00000100:00100000:2.0:1789760789.978344:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771250291502 time 771293647047 state 0x0 ref 1 00000100:00100000:2.0:1789760789.978357:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507376000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760789.978402:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23638016, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:2.0:1789760789.978411:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.978437:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760789.978442:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.978448:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.978468:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.023365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00002000:00010000:1.0:1789760790.023983:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23638016 -> 23642112 00010000:00010000:1.0:1789760790.023999:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024012:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024020:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024027:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760790.024032:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024046:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024066:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024073:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9a08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.024204:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b702d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507376000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45843us (46441us total) trans 4294973091 rc 0/0 00000100:00100000:1.0:1789760790.024215:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5801 00000100:00100000:1.0:1789760790.024224:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771250291502 time 771293647047 state 0x2 ref 0 00000100:00100000:3.0:1789760790.029799:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760790.030115:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376128 00000100:00100000:1.0:1789760790.030124:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376128 00000100:00100000:1.0:1789760790.030180:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771250291502 time 771293647047 state 0x0 ref 1 00000100:00100000:1.0:1789760790.030196:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771293907463 time 771293647047 state 0x1 ref 1 00000100:00100000:1.0:1789760790.030266:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771293907463 time 771345568795 state 0x0 ref 1 00000100:00100000:1.0:1789760790.030279:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760790.030324:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23642112, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xb807 00010000:00010000:1.0:1789760790.030333:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.030350:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760790.030355:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.030362:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.030376:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.102380:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00002000:00010000:2.0:1789760790.102739:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23642112 -> 23646208 00002000:00010000:2.0:1789760790.102744:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:2.0:1789760790.102748:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:2.0:1789760790.102760:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102773:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102789:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102796:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760790.102801:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102809:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102828:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.102837:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9a0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.102932:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72649us (73136us total) trans 4294973092 rc 0/0 00000100:00100000:2.0:1789760790.102941:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5802 00000100:00100000:2.0:1789760790.102951:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771293907463 time 771345568795 state 0x2 ref 0 00000100:00100000:3.0:1789760790.114026:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760790.114645:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376256 00000100:00100000:1.0:1789760790.114656:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376256 00000100:00100000:1.0:1789760790.114730:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771293907463 time 771345568795 state 0x0 ref 1 00000100:00100000:1.0:1789760790.114745:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771345829211 time 771345568795 state 0x1 ref 1 00000100:00100000:1.0:1789760790.114781:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771345829211 time 771430084313 state 0x0 ref 1 00000100:00100000:1.0:1789760790.114795:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760790.114845:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23646208, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:1.0:1789760790.114854:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.114870:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760790.114875:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.114882:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.114897:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.145938:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00002000:00010000:3.0:1789760790.146029:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:3.0:1789760790.146045:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146058:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146068:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146075:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.146080:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146090:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146109:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.146116:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9a16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.146216:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31418us (32196us total) trans 4294973093 rc 0/0 00000100:00100000:3.0:1789760790.146226:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5803 00000100:00100000:3.0:1789760790.146236:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771345829211 time 771430084313 state 0x2 ref 0 00000100:00100000:1.0:1789760790.161165:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760790.161294:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376384 00000100:00100000:2.0:1789760790.161305:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376384 00000100:00100000:2.0:1789760790.161362:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771345829211 time 771430084313 state 0x0 ref 1 00000100:00100000:2.0:1789760790.161378:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771430344729 time 771430084313 state 0x1 ref 1 00000100:00100000:2.0:1789760790.161411:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771430344729 time 771476714495 state 0x0 ref 1 00000100:00100000:2.0:1789760790.161433:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507376384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760790.161481:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23650304, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:2.0:1789760790.161490:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.161505:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760790.161510:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.161515:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.161530:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.212495:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00002000:00010000:3.0:1789760790.213514:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:3.0:1789760790.213533:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213550:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213560:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213568:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.213573:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213583:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213612:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.213621:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9a1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.213776:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507376384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52348us (52626us total) trans 4294973094 rc 0/0 00000100:00100000:3.0:1789760790.213787:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5804 00000100:00100000:3.0:1789760790.213797:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771430344729 time 771476714495 state 0x2 ref 0 00000100:00100000:2.0:1789760790.216983:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760790.217094:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376512 00000100:00100000:1.0:1789760790.217105:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376512 00000100:00100000:1.0:1789760790.217164:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771430344729 time 771476714495 state 0x0 ref 1 00000100:00100000:1.0:1789760790.217180:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771476974911 time 771476714495 state 0x1 ref 1 00000100:00100000:1.0:1789760790.217208:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771476974911 time 771532511624 state 0x0 ref 1 00000100:00100000:1.0:1789760790.217220:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760790.217266:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23654400, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:1.0:1789760790.217276:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.217291:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760790.217295:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.217302:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.217316:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.249073:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00002000:00010000:0.0:1789760790.249195:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:0.0:1789760790.249210:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249223:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249232:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249240:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.249245:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249261:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249282:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.249290:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9a24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.249392:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507376512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32162us (32405us total) trans 4294973095 rc 0/0 00000100:00100000:0.0:1789760790.249402:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5805 00000100:00100000:0.0:1789760790.249413:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771476974911 time 771532511624 state 0x2 ref 0 00000100:00100000:0.0:1789760790.255269:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.255580:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376640 00000100:00100000:3.0:1789760790.255591:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376640 00000100:00100000:3.0:1789760790.255642:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771476974911 time 771532511624 state 0x0 ref 1 00000100:00100000:3.0:1789760790.255657:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771532772040 time 771532511624 state 0x1 ref 1 00000100:00100000:3.0:1789760790.255755:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771532772040 time 771571057543 state 0x0 ref 1 00000100:00100000:3.0:1789760790.255767:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507376640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.255811:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23658496, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.255820:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.255835:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760790.255840:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.255846:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.255858:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.290621:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:2.0:1789760790.290641:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290654:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290664:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290677:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760790.290681:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290719:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290739:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.290748:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9a2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.290831:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507376640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35061us (35568us total) trans 4294973096 rc 0/0 00000100:00100000:2.0:1789760790.290841:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5806 00000100:00100000:2.0:1789760790.290852:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771532772040 time 771571057543 state 0x2 ref 0 00000001:00080000:1.0:1789760790.290995:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00000100:00100000:1.0:1789760790.298586:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.298716:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376768 00000100:00100000:3.0:1789760790.298728:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376768 00000100:00100000:3.0:1789760790.298781:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771532772040 time 771571057543 state 0x0 ref 1 00000100:00100000:3.0:1789760790.298796:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771571317959 time 771571057543 state 0x1 ref 1 00000100:00100000:3.0:1789760790.298827:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771571317959 time 771614130496 state 0x0 ref 1 00000100:00100000:3.0:1789760790.298839:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507376768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.298895:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23662592, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.298911:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.298925:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760790.298930:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.298936:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.298950:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760790.338363:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23662592 -> 23666688 00010000:00010000:3.0:1789760790.338382:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338395:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338406:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338413:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.338419:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338440:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338464:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.338473:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.338561:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507376768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39719us (39979us total) trans 4294973097 rc 0/0 00000100:00100000:3.0:1789760790.338575:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5807 00000100:00100000:3.0:1789760790.338584:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771571317959 time 771614130496 state 0x2 ref 0 00000001:00080000:2.0:1789760790.340606:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00000100:00100000:3.0:1789760790.342183:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760790.342275:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507376896 00000100:00100000:1.0:1789760790.342285:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507376896 00000100:00100000:1.0:1789760790.342336:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771571317959 time 771614130496 state 0x0 ref 1 00000100:00100000:1.0:1789760790.342351:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771614390912 time 771614130496 state 0x1 ref 1 00000100:00100000:1.0:1789760790.342384:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771614390912 time 771657686904 state 0x0 ref 1 00000100:00100000:1.0:1789760790.342396:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507376896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760790.342463:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23666688, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:1.0:1789760790.342472:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.342488:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760790.342492:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.342498:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.342512:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.370955:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00002000:00010000:0.0:1789760790.371109:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:0.0:1789760790.371125:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371139:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371148:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371160:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.371164:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371174:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371194:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371202:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9a39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.371294:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507376896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28895us (29113us total) trans 4294973098 rc 0/0 00000100:00100000:0.0:1789760790.371305:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5808 00000100:00100000:0.0:1789760790.371315:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771614390912 time 771657686904 state 0x2 ref 0 00000100:00100000:1.0:1789760790.375662:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760790.375810:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377024 00000100:00100000:2.0:1789760790.375817:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377024 00000100:00100000:2.0:1789760790.375861:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771614390912 time 771657686904 state 0x0 ref 1 00000100:00100000:2.0:1789760790.375879:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771657947320 time 771657686904 state 0x1 ref 1 00000100:00100000:2.0:1789760790.375964:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771657947320 time 771691268314 state 0x0 ref 1 00000100:00100000:2.0:1789760790.375974:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507377024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760790.376132:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23670784, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:2.0:1789760790.376141:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.376156:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760790.376161:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.376167:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.376179:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.405799:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:3.0:1789760790.405902:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:3.0:1789760790.405918:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405931:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405939:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405947:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.405952:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405962:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405981:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.405989:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9a40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.406087:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507377024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30108us (30430us total) trans 4294973099 rc 0/0 00000100:00100000:3.0:1789760790.406098:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5809 00000100:00100000:3.0:1789760790.406108:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771657947320 time 771691268314 state 0x2 ref 0 00000100:00100000:2.0:1789760790.409785:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.409884:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377152 00000100:00100000:3.0:1789760790.409893:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377152 00000100:00100000:3.0:1789760790.409946:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771657947320 time 771691268314 state 0x0 ref 1 00000100:00100000:3.0:1789760790.409961:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771691528730 time 771691268314 state 0x1 ref 1 00000100:00100000:3.0:1789760790.409990:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771691528730 time 771725293224 state 0x0 ref 1 00000100:00100000:3.0:1789760790.410003:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.410048:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23674880, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.410057:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.410072:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760790.410077:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.410084:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.410098:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.428813:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760790.443138:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00002000:00010000:2.0:1789760790.443237:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:2.0:1789760790.443253:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443265:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443274:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443287:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760790.443292:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443301:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443321:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.443341:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9a47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.443440:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33435us (33660us total) trans 4294973100 rc 0/0 00000100:00100000:2.0:1789760790.443451:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5810 00000100:00100000:2.0:1789760790.443468:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771691528730 time 771725293224 state 0x2 ref 0 00000100:00100000:2.0:1789760790.451883:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.452033:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377280 00000100:00100000:0.0:1789760790.452043:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377280 00000100:00100000:0.0:1789760790.452098:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771691528730 time 771725293224 state 0x0 ref 1 00000100:00100000:0.0:1789760790.452114:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771725553640 time 771725293224 state 0x1 ref 1 00000100:00100000:0.0:1789760790.452149:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771725553640 time 771767452621 state 0x0 ref 1 00000100:00100000:0.0:1789760790.452160:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507377280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760790.452207:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23678976, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:0.0:1789760790.452216:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.452230:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760790.452234:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.452241:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.452256:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.496208:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23678976 -> 23683072 00010000:00010000:0.0:1789760790.496229:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496242:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496252:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496260:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.496266:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496276:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496296:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.496304:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.496411:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507377280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44246us (44542us total) trans 4294973101 rc 0/0 00000100:00100000:0.0:1789760790.496587:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5811 00000100:00100000:0.0:1789760790.496597:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771725553640 time 771767452621 state 0x2 ref 0 00000001:00080000:0.0:1789760790.496636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00000100:00100000:1.0:1789760790.507776:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.509288:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377408 00000100:00100000:3.0:1789760790.509301:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377408 00000100:00100000:3.0:1789760790.509371:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771725553640 time 771767452621 state 0x0 ref 1 00000100:00100000:3.0:1789760790.509388:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771767713037 time 771767452621 state 0x1 ref 1 00000100:00100000:3.0:1789760790.509799:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771767713037 time 771824724163 state 0x0 ref 1 00000100:00100000:3.0:1789760790.509817:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507377408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.509877:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23683072, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.509886:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.509901:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760790.509906:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.509913:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.509927:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.573201:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00002000:00010000:0.0:1789760790.573397:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:0.0:1789760790.573413:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573436:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573444:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573464:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.573469:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573477:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573497:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.573504:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9a55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.573590:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507377408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63771us (65897us total) trans 4294973102 rc 0/0 00000100:00100000:0.0:1789760790.573600:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5812 00000100:00100000:0.0:1789760790.573610:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771767713037 time 771824724163 state 0x2 ref 0 00000100:00100000:2.0:1789760790.580554:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.580653:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377536 00000100:00100000:3.0:1789760790.580664:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377536 00000100:00100000:3.0:1789760790.580757:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771767713037 time 771824724163 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580773:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771824984579 time 771824724163 state 0x1 ref 1 00000100:00100000:3.0:1789760790.580795:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771824984579 time 771896099047 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580806:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507377536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.580853:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23687168, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.580861:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.580877:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760790.580881:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.580887:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.580900:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.613731:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:2.0:1789760790.613749:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613766:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613774:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613782:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760790.613786:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613795:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613815:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.613823:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9a5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.613906:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507377536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33096us (33357us total) trans 4294973103 rc 0/0 00000100:00100000:2.0:1789760790.613916:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5813 00000100:00100000:2.0:1789760790.613925:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771824984579 time 771896099047 state 0x2 ref 0 00000001:00080000:0.0:1789760790.614408:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00000100:00100000:2.0:1789760790.625572:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.625729:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377664 00000100:00100000:0.0:1789760790.625739:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377664 00000100:00100000:0.0:1789760790.625792:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771824984579 time 771896099047 state 0x0 ref 1 00000100:00100000:0.0:1789760790.625807:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771896359463 time 771896099047 state 0x1 ref 1 00000100:00100000:0.0:1789760790.625839:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771896359463 time 771941142252 state 0x0 ref 1 00000100:00100000:0.0:1789760790.625851:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760790.625900:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23691264, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:0.0:1789760790.625909:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.625925:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760790.625930:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.625936:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.625949:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.688032:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00002000:00010000:0.0:1789760790.688119:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:0.0:1789760790.688133:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688146:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688155:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688163:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.688167:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688175:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688197:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.688334:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.688474:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62620us (62910us total) trans 4294973104 rc 0/0 00000100:00100000:0.0:1789760790.688486:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5814 00000100:00100000:0.0:1789760790.688495:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771896359463 time 771941142252 state 0x2 ref 0 00000100:00100000:3.0:1789760790.691672:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.692149:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377792 00000100:00100000:0.0:1789760790.692161:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377792 00000100:00100000:2.0:1789760790.692316:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771896359463 time 771941142252 state 0x0 ref 1 00000100:00100000:2.0:1789760790.692332:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771941402668 time 771941142252 state 0x1 ref 1 00000100:00100000:2.0:1789760790.692367:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771941402668 time 772007670139 state 0x0 ref 1 00000100:00100000:2.0:1789760790.692380:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507377792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760790.692435:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23695360, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:2.0:1789760790.692444:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692467:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760790.692472:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.692479:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692493:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.748550:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538296960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760790.748550:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1051c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538297088:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760790.748618:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760790.748671:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760790.748719:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297088 00000100:00100000:2.0:1789760790.748728:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297088 00000100:00100000:1.0:1789760790.748729:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538296960 00000100:00100000:1.0:1789760790.748735:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538296960 00000100:00100000:2.0:1789760790.748779:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789760790.748789:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538297088:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760790.748811:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3834642432 avail 3832545280 00000100:00100000:1.0:1789760790.748837:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760790.748847:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538296960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760790.748859:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3834642432 avail 3832545280 00000100:00100000:2.0:1789760790.748859:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538297088:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760790.748871:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789760790.748901:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3464:x1876699538296960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.748908:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1789760790.748925:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1051c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538297088:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760790.749123:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1081e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538296960:0@lo:13:osp-pre-0-1.0 00002000:00010000:0.0:1789760790.755892:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:0.0:1789760790.755912:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755926:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755936:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755949:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.755955:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755964:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755983:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755991:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9a6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.756092:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507377792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63711us (64426us total) trans 4294973105 rc 0/0 00000100:00100000:0.0:1789760790.756102:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5815 00000100:00100000:0.0:1789760790.756112:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771941402668 time 772007670139 state 0x2 ref 0 00000001:00080000:3.0:1789760790.756737:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00000100:00100000:1.0:1789760790.765936:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760790.766110:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507377920 00000100:00100000:2.0:1789760790.766120:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507377920 00000100:00100000:2.0:1789760790.766175:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 771941402668 time 772007670139 state 0x0 ref 1 00000100:00100000:2.0:1789760790.766192:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772007930555 time 772007670139 state 0x1 ref 1 00000100:00100000:2.0:1789760790.766228:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772007930555 time 772081530779 state 0x0 ref 1 00000100:00100000:2.0:1789760790.766381:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760790.766444:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23699456, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:2.0:1789760790.766453:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.766480:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760790.766485:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.766492:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.766507:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.812833:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00002000:00010000:3.0:1789760790.812977:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23699456 -> 23703552 00010000:00010000:3.0:1789760790.812992:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813005:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813014:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813021:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.813026:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813035:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813054:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.813062:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9a71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.813157:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1081e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507377920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46914us (47225us total) trans 4294973106 rc 0/0 00000100:00100000:3.0:1789760790.813167:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5816 00000100:00100000:3.0:1789760790.813177:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772007930555 time 772081530779 state 0x2 ref 0 00000100:00100000:1.0:1789760790.817726:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.817857:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378048 00000100:00100000:0.0:1789760790.817867:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378048 00000100:00100000:0.0:1789760790.817932:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772007930555 time 772081530779 state 0x0 ref 1 00000100:00100000:0.0:1789760790.817948:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772081791195 time 772081530779 state 0x1 ref 1 00000100:00100000:0.0:1789760790.817979:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772081791195 time 772133282811 state 0x0 ref 1 00000100:00100000:0.0:1789760790.817996:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760790.818051:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23703552, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:0.0:1789760790.818061:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.818077:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760790.818081:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.818088:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.818103:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.884218:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:0.0:1789760790.884236:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884256:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884265:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884272:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760790.884276:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884286:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884305:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.884312:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.884540:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507378048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66542us (66855us total) trans 4294973107 rc 0/0 00000100:00100000:0.0:1789760790.884551:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5817 00000100:00100000:0.0:1789760790.884560:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772081791195 time 772133282811 state 0x2 ref 0 00000001:00080000:3.0:1789760790.884930:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00000100:00100000:1.0:1789760790.889320:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.889538:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378176 00000100:00100000:0.0:1789760790.889549:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378176 00000100:00100000:0.0:1789760790.889605:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772081791195 time 772133282811 state 0x0 ref 1 00000100:00100000:0.0:1789760790.889621:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772133543227 time 772133282811 state 0x1 ref 1 00000100:00100000:0.0:1789760790.889653:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772133543227 time 772204956288 state 0x0 ref 1 00000100:00100000:0.0:1789760790.889666:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760790.889755:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23707648, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:0.0:1789760790.889947:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.889965:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760790.889969:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.889976:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.889990:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.929540:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00002000:00010000:3.0:1789760790.929631:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23707648 -> 23711744 00010000:00010000:3.0:1789760790.929646:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929657:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929672:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929679:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.929684:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929720:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929740:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.929748:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9a7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.929832:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40162us (40516us total) trans 4294973108 rc 0/0 00000100:00100000:3.0:1789760790.929842:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5818 00000100:00100000:3.0:1789760790.929851:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772133543227 time 772204956288 state 0x2 ref 0 00000100:00100000:2.0:1789760790.933389:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760790.933480:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378304 00000100:00100000:3.0:1789760790.933490:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378304 00000100:00100000:3.0:1789760790.933542:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772133543227 time 772204956288 state 0x0 ref 1 00000100:00100000:3.0:1789760790.933556:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772205216704 time 772204956288 state 0x1 ref 1 00000100:00100000:3.0:1789760790.933632:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772205216704 time 772248934623 state 0x0 ref 1 00000100:00100000:3.0:1789760790.933645:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760790.933716:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23711744, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760790.933725:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.933740:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760790.933744:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.933750:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.933765:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.970854:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00002000:00010000:3.0:1789760790.971056:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:3.0:1789760790.971071:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971083:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971092:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971098:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760790.971103:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971112:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971130:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.971138:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.972160:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38509us (38774us total) trans 4294973109 rc 0/0 00000100:00100000:2.0:1789760790.972172:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5819 00000100:00100000:2.0:1789760790.972181:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772205216704 time 772248934623 state 0x2 ref 0 00000100:00100000:1.0:1789760790.975973:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760790.976144:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378432 00000100:00100000:0.0:1789760790.976153:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378432 00000100:00100000:0.0:1789760790.976207:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772205216704 time 772248934623 state 0x0 ref 1 00000100:00100000:0.0:1789760790.976222:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772249195039 time 772248934623 state 0x1 ref 1 00000100:00100000:0.0:1789760790.976255:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772249195039 time 772291558347 state 0x0 ref 1 00000100:00100000:0.0:1789760790.976267:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507378432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760790.976311:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23715840, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:0.0:1789760790.976319:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.976334:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760790.976339:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.976344:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.976362:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.003219:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00002000:00010000:1.0:1789760791.003313:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:1.0:1789760791.003326:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003338:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003352:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003359:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.003365:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003373:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003391:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.003398:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9a8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.003584:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507378432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27313us (27616us total) trans 4294973110 rc 0/0 00000100:00100000:1.0:1789760791.003595:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5820 00000100:00100000:1.0:1789760791.003604:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772249195039 time 772291558347 state 0x2 ref 0 00000100:00100000:1.0:1789760791.007432:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760791.007564:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378560 00000100:00100000:3.0:1789760791.007574:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378560 00000100:00100000:3.0:1789760791.007666:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772249195039 time 772291558347 state 0x0 ref 1 00000100:00100000:3.0:1789760791.007680:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772291818763 time 772291558347 state 0x1 ref 1 00000100:00100000:3.0:1789760791.007830:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772291818763 time 772323132660 state 0x0 ref 1 00000100:00100000:3.0:1789760791.007843:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760791.007897:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23719936, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xb807 00010000:00010000:3.0:1789760791.007906:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.007921:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760791.007927:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.007932:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.007945:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760791.008427:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760791.041508:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23719936 -> 23724032 00002000:00010000:0.0:1789760791.041517:0:8090:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:0.0:1789760791.041520:0:8090:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:0.0:1789760791.041538:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041557:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041567:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041574:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.041579:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041594:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041614:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.041632:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9a94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.041809:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507378560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33963us (34391us total) trans 4294973111 rc 0/0 00000100:00100000:0.0:1789760791.041825:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5821 00000100:00100000:0.0:1789760791.041835:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772291818763 time 772323132660 state 0x2 ref 0 00000001:00080000:2.0:1789760791.044993:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00000100:00100000:2.0:1789760791.045300:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.045370:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378688 00000100:00100000:0.0:1789760791.045380:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378688 00000100:00100000:0.0:1789760791.045442:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772291818763 time 772323132660 state 0x0 ref 1 00000100:00100000:0.0:1789760791.045465:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772323393076 time 772323132660 state 0x1 ref 1 00000100:00100000:0.0:1789760791.045492:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772323393076 time 772360795763 state 0x0 ref 1 00000100:00100000:0.0:1789760791.045504:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.045551:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23724032, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.045560:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.045574:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760791.045578:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.045585:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.045605:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.073830:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00002000:00010000:0.0:1789760791.073980:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23724032 -> 23728128 00010000:00010000:0.0:1789760791.073995:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074008:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074018:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074025:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.074030:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074040:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074060:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.074069:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9a9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.074168:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1051c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507378688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28661us (28868us total) trans 4294973112 rc 0/0 00000100:00100000:0.0:1789760791.074179:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5822 00000100:00100000:0.0:1789760791.074189:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772323393076 time 772360795763 state 0x2 ref 0 00000100:00100000:3.0:1789760791.077748:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.078063:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378816 00000100:00100000:0.0:1789760791.078074:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378816 00000100:00100000:0.0:1789760791.078134:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772323393076 time 772360795763 state 0x0 ref 1 00000100:00100000:0.0:1789760791.078150:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772361056179 time 772360795763 state 0x1 ref 1 00000100:00100000:0.0:1789760791.078173:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772361056179 time 772393476594 state 0x0 ref 1 00000100:00100000:0.0:1789760791.078185:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.078232:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23728128, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.078240:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.078257:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760791.078261:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.078269:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.078284:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.103272:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00002000:00010000:1.0:1789760791.103363:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:1.0:1789760791.103378:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103391:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103401:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103409:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.103414:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103433:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103461:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.103470:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9aa2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.103564:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25374us (25821us total) trans 4294973113 rc 0/0 00000100:00100000:1.0:1789760791.103575:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5823 00000100:00100000:1.0:1789760791.103585:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772361056179 time 772393476594 state 0x2 ref 0 00000100:00100000:3.0:1789760791.106635:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.106735:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507378944 00000100:00100000:0.0:1789760791.106745:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507378944 00000100:00100000:0.0:1789760791.106796:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772361056179 time 772393476594 state 0x0 ref 1 00000100:00100000:0.0:1789760791.106810:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772393737010 time 772393476594 state 0x1 ref 1 00000100:00100000:0.0:1789760791.106835:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772393737010 time 772422139122 state 0x0 ref 1 00000100:00100000:0.0:1789760791.106848:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.106903:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23732224, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.106911:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.106926:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760791.106931:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.106937:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.106951:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.145190:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00002000:00010000:3.0:1789760791.145293:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:3.0:1789760791.145308:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145321:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145331:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145339:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760791.145343:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145353:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145372:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.145380:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9aa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.145569:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507378944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38719us (38939us total) trans 4294973114 rc 0/0 00000100:00100000:3.0:1789760791.145579:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5824 00000100:00100000:3.0:1789760791.145589:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772393737010 time 772422139122 state 0x2 ref 0 00000100:00100000:3.0:1789760791.149761:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.149948:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379072 00000100:00100000:0.0:1789760791.149957:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379072 00000100:00100000:0.0:1789760791.150007:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772393737010 time 772422139122 state 0x0 ref 1 00000100:00100000:0.0:1789760791.150022:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772422399538 time 772422139122 state 0x1 ref 1 00000100:00100000:0.0:1789760791.150053:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772422399538 time 772465356210 state 0x0 ref 1 00000100:00100000:0.0:1789760791.150065:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507379072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.150112:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23736320, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.150121:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.150136:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760791.150140:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.150146:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.150159:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.180261:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:1.0:1789760791.180280:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180293:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180303:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180310:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.180315:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180324:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180341:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.180349:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9ab0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.180443:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507379072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30375us (30686us total) trans 4294973115 rc 0/0 00000100:00100000:1.0:1789760791.180459:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5825 00000100:00100000:1.0:1789760791.180469:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772422399538 time 772465356210 state 0x2 ref 0 00000001:00080000:3.0:1789760791.181524:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00000100:00100000:3.0:1789760791.183659:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.183961:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379200 00000100:00100000:1.0:1789760791.183970:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379200 00000100:00100000:1.0:1789760791.184021:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772422399538 time 772465356210 state 0x0 ref 1 00000100:00100000:1.0:1789760791.184038:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772465616626 time 772465356210 state 0x1 ref 1 00000100:00100000:1.0:1789760791.184064:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772465616626 time 772499367201 state 0x0 ref 1 00000100:00100000:1.0:1789760791.184077:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507379200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.184123:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23740416, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.184132:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.184148:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.184152:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.184158:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.184172:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.237486:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00002000:00010000:1.0:1789760791.237579:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:1.0:1789760791.237594:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237616:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237625:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237632:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.237636:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237645:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237664:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.237671:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ab7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.237789:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507379200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53711us (54133us total) trans 4294973116 rc 0/0 00000100:00100000:1.0:1789760791.237798:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5826 00000100:00100000:1.0:1789760791.237807:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772465616626 time 772499367201 state 0x2 ref 0 00000100:00100000:0.0:1789760791.243747:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760791.243898:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379328 00000100:00100000:3.0:1789760791.243909:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379328 00000100:00100000:3.0:1789760791.243961:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772465616626 time 772499367201 state 0x0 ref 1 00000100:00100000:3.0:1789760791.243977:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772499627617 time 772499367201 state 0x1 ref 1 00000100:00100000:3.0:1789760791.244011:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772499627617 time 772559313884 state 0x0 ref 1 00000100:00100000:3.0:1789760791.244025:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507379328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760791.244073:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23744512, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:3.0:1789760791.244082:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.244097:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760791.244102:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.244108:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.244122:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.260845:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538297216:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760791.260917:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760791.261071:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297216 00000100:00100000:3.0:1789760791.261079:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297216 00000100:00100000:3.0:1789760791.261122:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760791.261131:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538297216:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760791.261166:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760791.261260:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538297216:12345-0@lo:41:osp-pre-1-0.0 Request processed in 129us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760791.261266:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760791.261308:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1082ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538297216:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789760791.273877:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:3.0:1789760791.274000:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23744512 -> 23748608 00010000:00010000:3.0:1789760791.274014:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274027:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274036:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274042:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760791.274047:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274054:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274071:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.274078:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9abe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.274212:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b700700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507379328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30184us (30470us total) trans 4294973117 rc 0/0 00000100:00100000:3.0:1789760791.274222:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5827 00000100:00100000:3.0:1789760791.274232:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772499627617 time 772559313884 state 0x2 ref 0 00000100:00100000:3.0:1789760791.284831:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.287863:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379456 00000100:00100000:1.0:1789760791.287879:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379456 00000100:00100000:1.0:1789760791.288113:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772499627617 time 772559313884 state 0x0 ref 1 00000100:00100000:1.0:1789760791.288129:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772559574300 time 772559313884 state 0x1 ref 1 00000100:00100000:1.0:1789760791.288166:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772559574300 time 772603469247 state 0x0 ref 1 00000100:00100000:1.0:1789760791.288181:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507379456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.288233:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23748608, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.288242:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.288258:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.288263:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.288269:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.288283:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.333046:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:1.0:1789760791.333065:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333145:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333159:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333166:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.333171:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333180:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333201:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333209:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ac5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.333303:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507379456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45121us (48477us total) trans 4294973118 rc 0/0 00000100:00100000:1.0:1789760791.333313:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5828 00000100:00100000:1.0:1789760791.333323:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772559574300 time 772603469247 state 0x2 ref 0 00000001:00080000:3.0:1789760791.333819:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00000100:00100000:0.0:1789760791.336821:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.337053:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379584 00000100:00100000:1.0:1789760791.337062:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379584 00000100:00100000:1.0:1789760791.337123:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772559574300 time 772603469247 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337139:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772603729663 time 772603469247 state 0x1 ref 1 00000100:00100000:1.0:1789760791.337177:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772603729663 time 772652480040 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337190:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507379584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.337238:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23752704, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.337247:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337262:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.337267:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.337273:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337287:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.376629:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00002000:00010000:1.0:1789760791.376750:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23752704 -> 23756800 00010000:00010000:1.0:1789760791.376764:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376777:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376787:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376794:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.376800:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376808:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376826:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.376833:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9acc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.376920:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b701880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507379584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39727us (40103us total) trans 4294973119 rc 0/0 00000100:00100000:1.0:1789760791.376930:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5829 00000100:00100000:1.0:1789760791.376940:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772603729663 time 772652480040 state 0x2 ref 0 00000100:00100000:2.0:1789760791.382593:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760791.382744:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379712 00000100:00100000:3.0:1789760791.382754:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379712 00000100:00100000:3.0:1789760791.382811:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772603729663 time 772652480040 state 0x0 ref 1 00000100:00100000:3.0:1789760791.382825:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772652740456 time 772652480040 state 0x1 ref 1 00000100:00100000:3.0:1789760791.382941:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772652740456 time 772698242731 state 0x0 ref 1 00000100:00100000:3.0:1789760791.382955:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507379712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760791.383004:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23756800, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:3.0:1789760791.383012:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.383028:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760791.383032:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.383039:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.383053:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.426266:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00002000:00010000:0.0:1789760791.426436:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:0.0:1789760791.426452:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426465:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426480:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426487:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.426493:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426658:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426679:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.426727:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9ad3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.426963:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507379712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44005us (44373us total) trans 4294973120 rc 0/0 00000100:00100000:0.0:1789760791.426974:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5830 00000100:00100000:0.0:1789760791.426985:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772652740456 time 772698242731 state 0x2 ref 0 00000100:00100000:0.0:1789760791.430062:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.430102:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379840 00000100:00100000:1.0:1789760791.430113:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379840 00000100:00100000:1.0:1789760791.430286:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772652740456 time 772698242731 state 0x0 ref 1 00000100:00100000:1.0:1789760791.430303:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772698503147 time 772698242731 state 0x1 ref 1 00000100:00100000:1.0:1789760791.430484:0:8092:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772698503147 time 772745777811 state 0x0 ref 1 00000100:00100000:1.0:1789760791.430498:0:8092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507379840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.430545:0:8092:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23760896, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.430554:0:8092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.430566:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.430569:0:8092:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.430573:0:8092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.430583:0:8092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.451835:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760791.472356:0:8092:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23760896 -> 23764992 00010000:00010000:0.0:1789760791.472377:0:8092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472396:0:8092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472405:0:8092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472412:0:8092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.472427:0:8092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472437:0:8092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472468:0:8092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.472477:0:8092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9ada lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.472576:0:8092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507379840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42074us (42520us total) trans 4294973121 rc 0/0 00000100:00100000:0.0:1789760791.472587:0:8092:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5831 00000100:00100000:0.0:1789760791.472597:0:8092:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772698503147 time 772745777811 state 0x2 ref 0 00000001:00080000:2.0:1789760791.472780:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00000100:00100000:3.0:1789760791.477652:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.477771:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507379968 00000100:00100000:1.0:1789760791.477782:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507379968 00000100:00100000:1.0:1789760791.477848:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772698503147 time 772745777811 state 0x0 ref 1 00000100:00100000:1.0:1789760791.477865:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772746038227 time 772745777811 state 0x1 ref 1 00000100:00100000:1.0:1789760791.477898:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772746038227 time 772793200683 state 0x0 ref 1 00000100:00100000:1.0:1789760791.477912:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507379968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.477964:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23764992, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.477973:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477990:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.477995:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.478002:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.478018:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.526980:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00002000:00010000:0.0:1789760791.527150:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:0.0:1789760791.527166:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527178:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527188:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527195:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.527200:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527209:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527229:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.527237:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ae1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.527327:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104069f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507379968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49413us (49681us total) trans 4294973122 rc 0/0 00000100:00100000:0.0:1789760791.527337:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5832 00000100:00100000:0.0:1789760791.527347:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772746038227 time 772793200683 state 0x2 ref 0 00000100:00100000:1.0:1789760791.532430:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.532583:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380096 00000100:00100000:0.0:1789760791.532593:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380096 00000100:00100000:0.0:1789760791.532644:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772746038227 time 772793200683 state 0x0 ref 1 00000100:00100000:0.0:1789760791.532657:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772793461099 time 772793200683 state 0x1 ref 1 00000100:00100000:0.0:1789760791.532689:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772793461099 time 772847992001 state 0x0 ref 1 00000100:00100000:3.0:1789760791.533003:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760791.533053:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23769088, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:3.0:1789760791.533062:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.533077:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760791.533081:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.533087:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.533102:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.562195:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00002000:00010000:1.0:1789760791.562356:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:1.0:1789760791.562371:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562382:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562391:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562397:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.562402:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562411:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562440:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.562458:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9ae8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.562552:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1082ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29648us (30135us total) trans 4294973123 rc 0/0 00000100:00100000:1.0:1789760791.562562:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5833 00000100:00100000:1.0:1789760791.562572:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772793461099 time 772847992001 state 0x2 ref 0 00000100:00100000:3.0:1789760791.566462:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.566651:0:8092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380224 00000100:00100000:0.0:1789760791.566661:0:8092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380224 00000100:00100000:0.0:1789760791.566754:0:8092:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772793461099 time 772847992001 state 0x0 ref 1 00000100:00100000:0.0:1789760791.566923:0:8092:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772848252417 time 772847992001 state 0x1 ref 1 00000100:00100000:2.0:1789760791.567097:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772848252417 time 772882396701 state 0x0 ref 1 00000100:00100000:2.0:1789760791.567113:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760791.567291:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23773184, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:2.0:1789760791.567300:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.567315:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760791.567319:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.567325:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.567339:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.606557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00002000:00010000:1.0:1789760791.606858:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:1.0:1789760791.606874:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606888:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606897:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606905:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.606910:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606919:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606939:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606952:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9aef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.607170:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10406b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40055us (40714us total) trans 4294973124 rc 0/0 00000100:00100000:1.0:1789760791.607181:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5834 00000100:00100000:1.0:1789760791.607190:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772848252417 time 772882396701 state 0x2 ref 0 00000100:00100000:2.0:1789760791.610963:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.611044:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380352 00000100:00100000:0.0:1789760791.611115:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380352 00000100:00100000:0.0:1789760791.611169:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772848252417 time 772882396701 state 0x0 ref 1 00000100:00100000:0.0:1789760791.611184:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772882657117 time 772882396701 state 0x1 ref 1 00000100:00100000:0.0:1789760791.611244:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772882657117 time 772926547311 state 0x0 ref 1 00000100:00100000:0.0:1789760791.611256:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.611302:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23777280, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.611311:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.611332:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760791.611336:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.611342:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.611355:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.650722:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00002000:00010000:0.0:1789760791.650848:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:0.0:1789760791.650863:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650875:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650884:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650890:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760791.650894:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650903:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650920:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.650927:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9af6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.651016:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39757us (40059us total) trans 4294973125 rc 0/0 00000100:00100000:0.0:1789760791.651026:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5835 00000100:00100000:0.0:1789760791.651036:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772882657117 time 772926547311 state 0x2 ref 0 00000100:00100000:2.0:1789760791.655604:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.655807:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380480 00000100:00100000:1.0:1789760791.655819:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380480 00000100:00100000:1.0:1789760791.655874:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772882657117 time 772926547311 state 0x0 ref 1 00000100:00100000:1.0:1789760791.655891:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772926807727 time 772926547311 state 0x1 ref 1 00000100:00100000:1.0:1789760791.655923:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772926807727 time 772971225563 state 0x0 ref 1 00000100:00100000:1.0:1789760791.655935:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.655996:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23781376, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.656005:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656020:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760791.656025:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.656031:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656045:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.706471:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23781376 -> 23785472 00010000:00010000:1.0:1789760791.706490:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706505:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706515:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706522:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.706526:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706535:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706557:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.706565:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9afd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.706663:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507380480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50725us (51064us total) trans 4294973126 rc 0/0 00000100:00100000:1.0:1789760791.706674:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5836 00000100:00100000:1.0:1789760791.706682:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772926807727 time 772971225563 state 0x2 ref 0 00000001:00080000:0.0:1789760791.706854:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00000100:00100000:1.0:1789760791.709306:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760791.710735:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380608 00000100:00100000:2.0:1789760791.710748:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380608 00000100:00100000:2.0:1789760791.710808:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772926807727 time 772971225563 state 0x0 ref 1 00000100:00100000:2.0:1789760791.710824:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772971485979 time 772971225563 state 0x1 ref 1 00000100:00100000:2.0:1789760791.710848:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772971485979 time 773026151555 state 0x0 ref 1 00000100:00100000:2.0:1789760791.710861:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760791.710907:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23785472, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:2.0:1789760791.710917:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.710933:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760791.710937:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.710943:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.710956:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.744796:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:1.0:1789760791.744816:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744828:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744837:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744844:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.744849:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744858:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744880:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.744893:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.744990:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507380608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34126us (35687us total) trans 4294973127 rc 0/0 00000100:00100000:1.0:1789760791.745001:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5837 00000100:00100000:1.0:1789760791.745010:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772971485979 time 773026151555 state 0x2 ref 0 00000001:00080000:0.0:1789760791.745360:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00000100:00100000:3.0:1789760791.752144:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760791.752265:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380736 00000100:00100000:0.0:1789760791.752276:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380736 00000100:00100000:0.0:1789760791.752329:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 772971485979 time 773026151555 state 0x0 ref 1 00000100:00100000:0.0:1789760791.752345:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773026411971 time 773026151555 state 0x1 ref 1 00000100:00100000:0.0:1789760791.752551:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773026411971 time 773067852768 state 0x0 ref 1 00000100:00100000:0.0:1789760791.752566:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760791.752613:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23789568, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:0.0:1789760791.752622:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.752637:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760791.752642:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.752649:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.752663:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.772429:0:3464:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760791.772460:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c22d5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538297344:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760791.772535:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760791.772561:0:10721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297344 00000100:00100000:0.0:1789760791.772570:0:10721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297344 00000100:00100000:0.0:1789760791.772611:0:10721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:0.0:1789760791.772620:0:10721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538297344:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760791.772655:0:10721:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760791.772728:0:10721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3464:x1876699538297344:12345-0@lo:41:osp-pre-0-1.0 Request processed in 108us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760791.772736:0:10721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789760791.773776:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c22d5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3464:1876699538297344:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760791.798315:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00002000:00010000:1.0:1789760791.798387:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:1.0:1789760791.798402:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798425:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798435:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798442:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.798447:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798455:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798488:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.798495:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9b0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.798582:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104068e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46015us (46450us total) trans 4294973128 rc 0/0 00000100:00100000:1.0:1789760791.798591:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5838 00000100:00100000:1.0:1789760791.798600:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773026411971 time 773067852768 state 0x2 ref 0 00000100:00100000:2.0:1789760791.804651:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.805003:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380864 00000100:00100000:1.0:1789760791.805013:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380864 00000100:00100000:1.0:1789760791.805066:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773026411971 time 773067852768 state 0x0 ref 1 00000100:00100000:1.0:1789760791.805082:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773068113184 time 773067852768 state 0x1 ref 1 00000100:00100000:1.0:1789760791.805157:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773068113184 time 773120458920 state 0x0 ref 1 00000100:00100000:1.0:1789760791.805170:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.805217:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23793664, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.805227:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.805243:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760791.805248:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.805254:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.805266:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.845868:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00002000:00010000:1.0:1789760791.846023:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:1.0:1789760791.846039:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846052:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846061:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846068:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.846073:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846082:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846101:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846108:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.846202:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507380864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41029us (41557us total) trans 4294973129 rc 0/0 00000100:00100000:1.0:1789760791.846213:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5839 00000100:00100000:1.0:1789760791.846222:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773068113184 time 773120458920 state 0x2 ref 0 00000100:00100000:0.0:1789760791.850893:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.851002:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507380992 00000100:00100000:1.0:1789760791.851012:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507380992 00000100:00100000:1.0:1789760791.851068:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773068113184 time 773120458920 state 0x0 ref 1 00000100:00100000:1.0:1789760791.851084:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773120719336 time 773120458920 state 0x1 ref 1 00000100:00100000:1.0:1789760791.851116:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773120719336 time 773166419399 state 0x0 ref 1 00000100:00100000:1.0:1789760791.851130:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414d7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.851240:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23797760, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.851250:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851266:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760791.851271:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.851277:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851291:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.899012:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00002000:00010000:1.0:1789760791.899105:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:1.0:1789760791.899121:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899134:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899144:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899151:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.899156:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899165:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899185:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.899193:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.899325:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414d7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507380992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48192us (48437us total) trans 4294973130 rc 0/0 00000100:00100000:1.0:1789760791.899339:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5840 00000100:00100000:1.0:1789760791.899349:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773120719336 time 773166419399 state 0x2 ref 0 00000100:00100000:2.0:1789760791.908660:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.908948:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381120 00000100:00100000:1.0:1789760791.908958:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381120 00000100:00100000:1.0:1789760791.909012:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773120719336 time 773166419399 state 0x0 ref 1 00000100:00100000:1.0:1789760791.909028:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773166679815 time 773166419399 state 0x1 ref 1 00000100:00100000:1.0:1789760791.909049:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773166679815 time 773224352840 state 0x0 ref 1 00000100:00100000:1.0:1789760791.909062:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507381120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.909109:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23801856, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.909119:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.909134:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760791.909138:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.909144:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.909157:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.950751:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00002000:00010000:1.0:1789760791.950951:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:1.0:1789760791.950967:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.950981:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.950990:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.950996:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.951001:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.951009:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.951027:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.951036:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.951128:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507381120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42064us (42472us total) trans 4294973131 rc 0/0 00000100:00100000:1.0:1789760791.951138:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5841 00000100:00100000:1.0:1789760791.951148:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773166679815 time 773224352840 state 0x2 ref 0 00000100:00100000:2.0:1789760791.955372:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.957977:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381248 00000100:00100000:1.0:1789760791.957990:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381248 00000100:00100000:1.0:1789760791.958054:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773166679815 time 773224352840 state 0x0 ref 1 00000100:00100000:1.0:1789760791.958075:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773224613256 time 773224352840 state 0x1 ref 1 00000100:00100000:1.0:1789760791.958258:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773224613256 time 773273559738 state 0x0 ref 1 00000100:00100000:1.0:1789760791.958274:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507381248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.958325:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23805952, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.958334:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.958350:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760791.958355:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.958361:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.958375:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.991313:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23805952 -> 23810048 00010000:00010000:1.0:1789760791.991335:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991349:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991363:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991370:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760791.991375:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991385:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991404:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.991422:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.991588:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507381248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33311us (36220us total) trans 4294973132 rc 0/0 00000100:00100000:1.0:1789760791.991600:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5842 00000100:00100000:1.0:1789760791.991610:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773224613256 time 773273559738 state 0x2 ref 0 00000100:00100000:3.0:1789760791.995307:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760791.995393:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381376 00000100:00100000:1.0:1789760791.995403:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381376 00000100:00100000:1.0:1789760791.995476:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773224613256 time 773273559738 state 0x0 ref 1 00000100:00100000:1.0:1789760791.995491:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773273820154 time 773273559738 state 0x1 ref 1 00000100:00100000:1.0:1789760791.995522:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773273820154 time 773310825021 state 0x0 ref 1 00000100:00100000:1.0:1789760791.995534:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507381376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760791.995580:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23810048, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xb807 00010000:00010000:1.0:1789760791.995589:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.995604:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760791.995609:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.995615:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.995630:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.996789:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00000001:00080000:3.0:1789760792.034529:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00002000:00010000:1.0:1789760792.034641:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23810048 -> 23814144 00002000:00010000:1.0:1789760792.034646:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:1.0:1789760792.034649:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:1.0:1789760792.034661:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034675:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034685:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034721:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.034727:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034736:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034757:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.034764:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.034845:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507381376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39309us (39544us total) trans 4294973133 rc 0/0 00000100:00100000:1.0:1789760792.034865:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5843 00000100:00100000:1.0:1789760792.034875:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773273820154 time 773310825021 state 0x2 ref 0 00000100:00100000:3.0:1789760792.038136:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.038226:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381504 00000100:00100000:1.0:1789760792.038235:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381504 00000100:00100000:1.0:1789760792.038287:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773273820154 time 773310825021 state 0x0 ref 1 00000100:00100000:1.0:1789760792.038302:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773311085437 time 773310825021 state 0x1 ref 1 00000100:00100000:1.0:1789760792.038332:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773311085437 time 773353635814 state 0x0 ref 1 00000100:00100000:1.0:1789760792.038347:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507381504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.038395:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23814144, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.038404:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.038432:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760792.038436:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.038443:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.038457:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.070070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00002000:00010000:2.0:1789760792.070183:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:2.0:1789760792.070199:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070213:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070223:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070230:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbe00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.070234:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070243:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070262:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.070270:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cd149f4/0x9638aab0164b9b35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.070356:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507381504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32005us (32223us total) trans 4294973134 rc 0/0 00000100:00100000:2.0:1789760792.070367:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5844 00000100:00100000:2.0:1789760792.070377:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773311085437 time 773353635814 state 0x2 ref 0 00000100:00100000:3.0:1789760792.074299:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.074449:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381632 00000100:00100000:1.0:1789760792.074459:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381632 00000100:00100000:1.0:1789760792.074525:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773311085437 time 773353635814 state 0x0 ref 1 00000100:00100000:1.0:1789760792.074541:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773353896230 time 773353635814 state 0x1 ref 1 00000100:00100000:1.0:1789760792.074571:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773353896230 time 773389874739 state 0x0 ref 1 00000100:00100000:1.0:1789760792.074583:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507381632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.074631:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23818240, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.074640:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.074656:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760792.074660:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.074667:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.074680:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.106684:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00002000:00010000:2.0:1789760792.106810:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:2.0:1789760792.106825:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106837:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106845:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106852:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.106857:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106866:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106890:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.106898:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.106991:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507381632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32404us (32696us total) trans 4294973135 rc 0/0 00000100:00100000:2.0:1789760792.107001:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5845 00000100:00100000:2.0:1789760792.107011:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773353896230 time 773389874739 state 0x2 ref 0 00000100:00100000:2.0:1789760792.113675:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.113897:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381760 00000100:00100000:1.0:1789760792.113907:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381760 00000100:00100000:1.0:1789760792.113959:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773353896230 time 773389874739 state 0x0 ref 1 00000100:00100000:1.0:1789760792.113975:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773390135155 time 773389874739 state 0x1 ref 1 00000100:00100000:1.0:1789760792.114005:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773390135155 time 773429308588 state 0x0 ref 1 00000100:00100000:1.0:1789760792.114016:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507381760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.114063:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23822336, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.114071:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.114085:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760792.114090:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.114096:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.114110:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.150281:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00002000:00010000:0.0:1789760792.150365:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:0.0:1789760792.150380:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.150394:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.150402:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.150982:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760792.150988:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.151000:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.151020:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.151027:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d9a25f/0x9638aab0164b9b43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.151171:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507381760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37150us (37500us total) trans 4294973136 rc 0/0 00000100:00100000:0.0:1789760792.151182:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5846 00000100:00100000:0.0:1789760792.151192:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773390135155 time 773429308588 state 0x2 ref 0 00000100:00100000:3.0:1789760792.155002:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.155362:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507381888 00000100:00100000:2.0:1789760792.155373:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507381888 00000100:00100000:2.0:1789760792.155453:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773390135155 time 773429308588 state 0x0 ref 1 00000100:00100000:2.0:1789760792.155470:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773429569004 time 773429308588 state 0x1 ref 1 00000100:00100000:2.0:1789760792.155499:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773429569004 time 773470802691 state 0x0 ref 1 00000100:00100000:2.0:1789760792.155512:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507381888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.155563:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23826432, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.155572:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.155593:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.155597:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.155603:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.155616:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760792.192355:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:1.0:1789760792.192373:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192386:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192396:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192403:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.192424:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192434:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192459:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192468:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9b4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.192559:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507381888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37045us (37562us total) trans 4294973137 rc 0/0 00000100:00100000:1.0:1789760792.192569:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5847 00000100:00100000:1.0:1789760792.192578:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773429569004 time 773470802691 state 0x2 ref 0 00000001:00080000:3.0:1789760792.193245:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00000100:00100000:3.0:1789760792.201077:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.201476:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382016 00000100:00100000:2.0:1789760792.201487:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382016 00000100:00100000:2.0:1789760792.201546:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773429569004 time 773470802691 state 0x0 ref 1 00000100:00100000:2.0:1789760792.201561:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773471063107 time 773470802691 state 0x1 ref 1 00000100:00100000:2.0:1789760792.201595:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773471063107 time 773516898210 state 0x0 ref 1 00000100:00100000:2.0:1789760792.201607:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507382016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.201656:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23830528, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.201665:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.201680:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.201685:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.201734:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.201756:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.275024:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00002000:00010000:2.0:1789760792.275477:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:2.0:1789760792.275493:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275505:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275513:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275520:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.275526:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275535:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275554:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.275560:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.275640:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507382016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 74030us (74568us total) trans 4294973138 rc 0/0 00000100:00100000:2.0:1789760792.275650:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5848 00000100:00100000:2.0:1789760792.275661:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773471063107 time 773516898210 state 0x2 ref 0 00000100:00100000:2.0:1789760792.279125:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.279734:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382144 00000100:00100000:0.0:1789760792.279745:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382144 00000100:00100000:0.0:1789760792.279805:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773471063107 time 773516898210 state 0x0 ref 1 00000100:00100000:0.0:1789760792.279821:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773517158626 time 773516898210 state 0x1 ref 1 00000100:00100000:0.0:1789760792.279846:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773517158626 time 773595150562 state 0x0 ref 1 00000100:00100000:0.0:1789760792.279858:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507382144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.279903:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23834624, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.279912:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279927:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760792.279932:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.279938:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279952:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.312761:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:2.0:1789760792.312780:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312792:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312802:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312810:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.312815:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312824:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312844:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.312857:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.312953:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507382144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33092us (33834us total) trans 4294973139 rc 0/0 00000100:00100000:2.0:1789760792.312964:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5849 00000100:00100000:2.0:1789760792.312973:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773517158626 time 773595150562 state 0x2 ref 0 00000001:00080000:3.0:1789760792.313535:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00000100:00100000:1.0:1789760792.317386:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.317529:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382272 00000100:00100000:2.0:1789760792.317538:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382272 00000100:00100000:2.0:1789760792.317589:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773517158626 time 773595150562 state 0x0 ref 1 00000100:00100000:2.0:1789760792.317603:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773595410978 time 773595150562 state 0x1 ref 1 00000100:00100000:2.0:1789760792.317628:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773595410978 time 773632931161 state 0x0 ref 1 00000100:00100000:2.0:1789760792.317641:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507382272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.317687:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23838720, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.317733:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.317750:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.317754:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.317761:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.317782:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.345272:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00002000:00010000:1.0:1789760792.345613:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:1.0:1789760792.345628:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345639:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345648:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345655:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.345659:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345668:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345688:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.345735:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9b5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.345826:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507382272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28182us (28444us total) trans 4294973140 rc 0/0 00000100:00100000:1.0:1789760792.345836:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5850 00000100:00100000:1.0:1789760792.345846:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773595410978 time 773632931161 state 0x2 ref 0 00000100:00100000:1.0:1789760792.348777:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.348890:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382400 00000100:00100000:2.0:1789760792.348899:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382400 00000100:00100000:2.0:1789760792.348956:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773595410978 time 773632931161 state 0x0 ref 1 00000100:00100000:2.0:1789760792.348972:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773633191577 time 773632931161 state 0x1 ref 1 00000100:00100000:2.0:1789760792.349039:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773633191577 time 773664341179 state 0x0 ref 1 00000100:00100000:2.0:1789760792.349052:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507382400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.349100:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23842816, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.349110:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.349125:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.349130:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.349136:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.349146:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.376630:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00002000:00010000:2.0:1789760792.376920:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23842816 -> 23846912 00010000:00010000:2.0:1789760792.376936:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.376949:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.376958:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.376965:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.376970:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.376978:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.376998:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.377006:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.377137:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507382400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28082us (28443us total) trans 4294973141 rc 0/0 00000100:00100000:2.0:1789760792.377148:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5851 00000100:00100000:2.0:1789760792.377159:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773633191577 time 773664341179 state 0x2 ref 0 00000100:00100000:2.0:1789760792.391404:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.391610:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382528 00000100:00100000:0.0:1789760792.391620:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382528 00000100:00100000:0.0:1789760792.391678:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773633191577 time 773664341179 state 0x0 ref 1 00000100:00100000:0.0:1789760792.391714:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773664601595 time 773664341179 state 0x1 ref 1 00000100:00100000:0.0:1789760792.391787:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773664601595 time 773707090160 state 0x0 ref 1 00000100:00100000:0.0:1789760792.391801:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507382528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.391847:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23846912, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.391856:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.391872:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760792.391877:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.391884:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.391898:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.449178:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:3.0:1789760792.449502:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:3.0:1789760792.449518:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449531:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449539:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449545:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760792.449551:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449565:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449588:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.449596:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9b6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.449732:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507382528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57889us (58294us total) trans 4294973142 rc 0/0 00000100:00100000:3.0:1789760792.449743:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5852 00000100:00100000:3.0:1789760792.449753:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773664601595 time 773707090160 state 0x2 ref 0 00000100:00100000:3.0:1789760792.456980:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.457079:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382656 00000100:00100000:1.0:1789760792.457089:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382656 00000100:00100000:1.0:1789760792.457145:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773664601595 time 773707090160 state 0x0 ref 1 00000100:00100000:1.0:1789760792.457161:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773707350576 time 773707090160 state 0x1 ref 1 00000100:00100000:1.0:1789760792.457191:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773707350576 time 773772494331 state 0x0 ref 1 00000100:00100000:1.0:1789760792.457204:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507382656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.457249:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23851008, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.457258:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457274:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760792.457279:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.457286:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457298:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.475805:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760792.484539:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:2.0:1789760792.484622:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:2.0:1789760792.484637:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484651:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484660:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484667:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.484671:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484680:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484758:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.484767:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9b74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.484895:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507382656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27689us (27919us total) trans 4294973143 rc 0/0 00000100:00100000:2.0:1789760792.484910:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5853 00000100:00100000:2.0:1789760792.484920:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773707350576 time 773772494331 state 0x2 ref 0 00000100:00100000:1.0:1789760792.488114:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.488210:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382784 00000100:00100000:2.0:1789760792.488219:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382784 00000100:00100000:2.0:1789760792.488273:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773707350576 time 773772494331 state 0x0 ref 1 00000100:00100000:2.0:1789760792.488288:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773772754747 time 773772494331 state 0x1 ref 1 00000100:00100000:2.0:1789760792.488562:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773772754747 time 773803864588 state 0x0 ref 1 00000100:00100000:2.0:1789760792.488576:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507382784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.488624:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23855104, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.488633:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.488648:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.488652:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.488658:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.488671:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.519307:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23855104 -> 23859200 00010000:00010000:2.0:1789760792.519328:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519350:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519359:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519365:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.519370:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519379:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519403:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.519418:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9b7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.519631:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507382784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31051us (31520us total) trans 4294973144 rc 0/0 00000100:00100000:2.0:1789760792.519641:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5854 00000100:00100000:2.0:1789760792.519652:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773772754747 time 773803864588 state 0x2 ref 0 00000001:00080000:3.0:1789760792.519783:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00000100:00100000:3.0:1789760792.522440:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.522542:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507382912 00000100:00100000:0.0:1789760792.522551:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507382912 00000100:00100000:0.0:1789760792.522595:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773772754747 time 773803864588 state 0x0 ref 1 00000100:00100000:0.0:1789760792.522606:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773804125004 time 773803864588 state 0x1 ref 1 00000100:00100000:0.0:1789760792.522630:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773804125004 time 773837934501 state 0x0 ref 1 00000100:00100000:0.0:1789760792.522639:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507382912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.522679:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23859200, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.522686:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.522811:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760792.522814:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.522819:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.522829:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.547829:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:3.0:1789760792.547849:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547862:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547878:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547885:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760792.547890:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547900:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547919:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.547927:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.548030:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507382912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25385us (25593us total) trans 4294973145 rc 0/0 00000100:00100000:3.0:1789760792.548040:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5855 00000100:00100000:3.0:1789760792.548051:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773804125004 time 773837934501 state 0x2 ref 0 00000001:00080000:1.0:1789760792.548133:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00000100:00100000:1.0:1789760792.553080:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.553171:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383040 00000100:00100000:2.0:1789760792.553181:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383040 00000100:00100000:2.0:1789760792.553236:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773804125004 time 773837934501 state 0x0 ref 1 00000100:00100000:2.0:1789760792.553250:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773838194917 time 773837934501 state 0x1 ref 1 00000100:00100000:2.0:1789760792.553283:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773838194917 time 773868586796 state 0x0 ref 1 00000100:00100000:2.0:1789760792.553295:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.553344:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23863296, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.553353:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.553368:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760792.553373:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.553379:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.553392:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.586224:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:2.0:1789760792.586248:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586262:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586277:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586290:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.586296:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586305:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586326:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.586334:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.586438:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507383040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33139us (33361us total) trans 4294973146 rc 0/0 00000100:00100000:2.0:1789760792.586456:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5856 00000100:00100000:2.0:1789760792.586465:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773838194917 time 773868586796 state 0x2 ref 0 00000001:00080000:1.0:1789760792.586653:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00000100:00100000:2.0:1789760792.591564:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.591983:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383168 00000100:00100000:0.0:1789760792.591994:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383168 00000100:00100000:0.0:1789760792.592050:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773838194917 time 773868586796 state 0x0 ref 1 00000100:00100000:0.0:1789760792.592065:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773868847212 time 773868586796 state 0x1 ref 1 00000100:00100000:0.0:1789760792.592131:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773868847212 time 773907433835 state 0x0 ref 1 00000100:00100000:0.0:1789760792.592144:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507383168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.592198:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23867392, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.592208:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.592223:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760792.592228:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.592235:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.592248:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.628378:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00002000:00010000:1.0:1789760792.628560:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:1.0:1789760792.628574:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628585:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628594:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628601:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.628605:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628613:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628631:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.628640:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9b90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.628760:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507383168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36613us (37200us total) trans 4294973147 rc 0/0 00000100:00100000:1.0:1789760792.628769:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5857 00000100:00100000:1.0:1789760792.628778:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773868847212 time 773907433835 state 0x2 ref 0 00000100:00100000:3.0:1789760792.635172:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.635240:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383296 00000100:00100000:0.0:1789760792.635250:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383296 00000100:00100000:0.0:1789760792.635303:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773868847212 time 773907433835 state 0x0 ref 1 00000100:00100000:0.0:1789760792.635318:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773907694251 time 773907433835 state 0x1 ref 1 00000100:00100000:0.0:1789760792.635347:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773907694251 time 773950650525 state 0x0 ref 1 00000100:00100000:0.0:1789760792.635361:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.635404:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23871488, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.635423:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.635438:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760792.635442:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.635450:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.635465:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.699256:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00002000:00010000:0.0:1789760792.701589:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:0.0:1789760792.701611:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701624:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701636:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701644:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760792.701649:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701659:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701685:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.701725:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9b97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.701834:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 66469us (66665us total) trans 4294973148 rc 0/0 00000100:00100000:0.0:1789760792.701845:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5858 00000100:00100000:0.0:1789760792.701856:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773907694251 time 773950650525 state 0x2 ref 0 00000100:00100000:0.0:1789760792.708480:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.708560:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383424 00000100:00100000:2.0:1789760792.708570:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383424 00000100:00100000:2.0:1789760792.708630:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773907694251 time 773950650525 state 0x0 ref 1 00000100:00100000:2.0:1789760792.708648:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773950910941 time 773950650525 state 0x1 ref 1 00000100:00100000:2.0:1789760792.708681:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773950910941 time 774023983984 state 0x0 ref 1 00000100:00100000:2.0:1789760792.708718:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507383424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.708766:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23875584, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.708775:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.708791:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760792.708796:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.708802:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.708817:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.755743:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00002000:00010000:3.0:1789760792.755979:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23875584 -> 23879680 00010000:00010000:3.0:1789760792.755995:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756009:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756020:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756028:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760792.756033:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756043:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756064:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756072:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9b9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.756168:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507383424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47471us (47692us total) trans 4294973149 rc 0/0 00000100:00100000:3.0:1789760792.756179:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5859 00000100:00100000:3.0:1789760792.756189:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773950910941 time 774023983984 state 0x2 ref 0 00000100:00100000:3.0:1789760792.759993:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.760181:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383552 00000100:00100000:1.0:1789760792.760373:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383552 00000100:00100000:1.0:1789760792.760444:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 773950910941 time 774023983984 state 0x0 ref 1 00000100:00100000:1.0:1789760792.760460:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774024244400 time 774023983984 state 0x1 ref 1 00000100:00100000:1.0:1789760792.760491:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774024244400 time 774075794663 state 0x0 ref 1 00000100:00100000:1.0:1789760792.760503:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.760544:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23879680, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.760552:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.760568:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760792.760573:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.760580:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.760595:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.806236:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:2.0:1789760792.806257:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.806270:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.806277:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00010000:00010000:2.0:1789760792.806348:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.806363:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dba00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760792.806368:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.806378:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.806396:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.806414:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bc0d8a1/0x9638aab0164b9ba5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.806546:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46040us (46558us total) trans 4294973150 rc 0/0 00000100:00100000:2.0:1789760792.806556:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5860 00000100:00100000:2.0:1789760792.806566:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774024244400 time 774075794663 state 0x2 ref 0 00000100:00100000:3.0:1789760792.811836:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.812030:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383680 00000100:00100000:0.0:1789760792.812040:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383680 00000100:00100000:0.0:1789760792.812163:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774024244400 time 774075794663 state 0x0 ref 1 00000100:00100000:0.0:1789760792.812178:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774076055079 time 774075794663 state 0x1 ref 1 00000100:00100000:0.0:1789760792.812246:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774076055079 time 774127548957 state 0x0 ref 1 00000100:00100000:0.0:1789760792.812260:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.812306:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23883776, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.812325:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.812340:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760792.812344:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.812350:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.812363:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.844369:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00002000:00010000:1.0:1789760792.844516:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:1.0:1789760792.844531:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844543:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844551:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844558:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.844562:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844571:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844590:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.844599:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b353f/0x9638aab0164b9bac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.844732:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32433us (32865us total) trans 4294973151 rc 0/0 00000100:00100000:1.0:1789760792.844743:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5861 00000100:00100000:1.0:1789760792.844753:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774076055079 time 774127548957 state 0x2 ref 0 00000100:00100000:0.0:1789760792.848828:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760792.849058:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383808 00000100:00100000:2.0:1789760792.849069:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383808 00000100:00100000:2.0:1789760792.849119:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774076055079 time 774127548957 state 0x0 ref 1 00000100:00100000:2.0:1789760792.849134:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774127809373 time 774127548957 state 0x1 ref 1 00000100:00100000:2.0:1789760792.849160:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774127809373 time 774164463817 state 0x0 ref 1 00000100:00100000:2.0:1789760792.849175:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507383808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760792.849220:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23887872, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:2.0:1789760792.849229:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.849244:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760792.849249:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.849255:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.849269:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.900191:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00002000:00010000:1.0:1789760792.900335:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:1.0:1789760792.900350:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900363:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900371:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900377:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.900383:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900392:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900421:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.900434:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9bb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.900529:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507383808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51350us (51706us total) trans 4294973152 rc 0/0 00000100:00100000:1.0:1789760792.900542:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5862 00000100:00100000:1.0:1789760792.900552:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774127809373 time 774164463817 state 0x2 ref 0 00000100:00100000:3.0:1789760792.904103:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760792.904212:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507383936 00000100:00100000:0.0:1789760792.904222:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507383936 00000100:00100000:0.0:1789760792.904277:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774127809373 time 774164463817 state 0x0 ref 1 00000100:00100000:0.0:1789760792.904293:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774164724233 time 774164463817 state 0x1 ref 1 00000100:00100000:0.0:1789760792.904326:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774164724233 time 774219629076 state 0x0 ref 1 00000100:00100000:0.0:1789760792.904338:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760792.904386:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23891968, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:0.0:1789760792.904395:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.904434:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760792.904490:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.904497:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.904507:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.962220:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00002000:00010000:1.0:1789760792.962369:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23891968 -> 23896064 00010000:00010000:1.0:1789760792.962385:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962399:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962422:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962431:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760792.962436:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962456:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962482:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.962491:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061902467/0x9638aab0164b9bba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.962589:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507383936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58247us (58491us total) trans 4294973153 rc 0/0 00000100:00100000:1.0:1789760792.962600:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5863 00000100:00100000:1.0:1789760792.962610:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774164724233 time 774219629076 state 0x2 ref 0 00000100:00100000:3.0:1789760792.965926:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760792.966076:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384064 00000100:00100000:1.0:1789760792.966086:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384064 00000100:00100000:1.0:1789760792.966139:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774164724233 time 774219629076 state 0x0 ref 1 00000100:00100000:1.0:1789760792.966154:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774219889492 time 774219629076 state 0x1 ref 1 00000100:00100000:1.0:1789760792.966172:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774219889492 time 774281476553 state 0x0 ref 1 00000100:00100000:1.0:1789760792.966183:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760792.966234:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23896064, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760792.966243:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.966259:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760792.966264:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.966271:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.966285:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.994209:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00002000:00010000:3.0:1789760792.994365:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:3.0:1789760792.994380:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994394:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994414:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994422:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760792.994427:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994436:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994466:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.994474:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9bc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.994572:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28384us (28650us total) trans 4294973154 rc 0/0 00000100:00100000:3.0:1789760792.994584:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5864 00000100:00100000:3.0:1789760792.994593:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774219889492 time 774281476553 state 0x2 ref 0 00000100:00100000:3.0:1789760792.999842:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.000334:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384192 00000100:00100000:1.0:1789760793.000343:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384192 00000100:00100000:1.0:1789760793.000399:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774219889492 time 774281476553 state 0x0 ref 1 00000100:00100000:1.0:1789760793.000425:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774281736969 time 774281476553 state 0x1 ref 1 00000100:00100000:1.0:1789760793.000461:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774281736969 time 774315764086 state 0x0 ref 1 00000100:00100000:1.0:1789760793.000473:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.000519:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23900160, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xb807 00010000:00010000:1.0:1789760793.000528:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.000543:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.000547:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.000554:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.000567:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.046289:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00002000:00010000:1.0:1789760793.046386:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23900160 -> 23904256 00002000:00010000:1.0:1789760793.046390:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:1.0:1789760793.046393:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:1.0:1789760793.046415:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046428:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046497:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046506:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.046511:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046521:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046540:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046548:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.046677:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46202us (46842us total) trans 4294973155 rc 0/0 00000100:00100000:1.0:1789760793.046688:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5865 00000100:00100000:1.0:1789760793.046736:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774281736969 time 774315764086 state 0x2 ref 0 00000100:00100000:0.0:1789760793.062655:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.062780:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384320 00000100:00100000:1.0:1789760793.062790:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384320 00000100:00100000:1.0:1789760793.062900:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774281736969 time 774315764086 state 0x0 ref 1 00000100:00100000:1.0:1789760793.062915:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774316024502 time 774315764086 state 0x1 ref 1 00000100:00100000:1.0:1789760793.062945:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774316024502 time 774378248217 state 0x0 ref 1 00000100:00100000:1.0:1789760793.062958:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.063007:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23904256, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.063017:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.063033:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.063037:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.063044:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.063058:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760793.063544:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760793.084067:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00002000:00010000:1.0:1789760793.084158:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:1.0:1789760793.084174:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084188:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084199:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084207:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.084212:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084223:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084245:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.084252:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.084393:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 21432us (21744us total) trans 4294973156 rc 0/0 00000100:00100000:1.0:1789760793.084417:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5866 00000100:00100000:1.0:1789760793.084426:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774316024502 time 774378248217 state 0x2 ref 0 00000100:00100000:0.0:1789760793.096127:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760793.096217:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384448 00000100:00100000:3.0:1789760793.096228:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384448 00000100:00100000:3.0:1789760793.096281:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774316024502 time 774378248217 state 0x0 ref 1 00000100:00100000:3.0:1789760793.096297:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774378508633 time 774378248217 state 0x1 ref 1 00000100:00100000:3.0:1789760793.096369:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774378508633 time 774411672305 state 0x0 ref 1 00000100:00100000:3.0:1789760793.096383:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760793.096437:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23908352, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:3.0:1789760793.096454:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.096471:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760793.096475:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.096481:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.096500:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.126557:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00002000:00010000:1.0:1789760793.126645:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:1.0:1789760793.126661:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126675:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126685:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126725:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.126730:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126739:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126758:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.126766:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9bd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.126971:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30585us (30847us total) trans 4294973157 rc 0/0 00000100:00100000:1.0:1789760793.126981:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5867 00000100:00100000:1.0:1789760793.126991:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774378508633 time 774411672305 state 0x2 ref 0 00000100:00100000:2.0:1789760793.134072:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.134155:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384576 00000100:00100000:1.0:1789760793.134164:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384576 00000100:00100000:1.0:1789760793.134214:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774378508633 time 774411672305 state 0x0 ref 1 00000100:00100000:1.0:1789760793.134229:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774411932721 time 774411672305 state 0x1 ref 1 00000100:00100000:1.0:1789760793.134257:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774411932721 time 774449560649 state 0x0 ref 1 00000100:00100000:1.0:1789760793.134268:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.134309:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23912448, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.134318:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.134332:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.134337:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.134342:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.134355:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.168310:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:1.0:1789760793.168332:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168345:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168355:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168363:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.168368:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168377:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168398:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.168417:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9bdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.168517:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507384576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34244us (34447us total) trans 4294973158 rc 0/0 00000100:00100000:1.0:1789760793.168531:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5868 00000100:00100000:1.0:1789760793.168541:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774411932721 time 774449560649 state 0x2 ref 0 00000001:00080000:0.0:1789760793.169069:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00000100:00100000:3.0:1789760793.177688:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.178786:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384704 00000100:00100000:1.0:1789760793.178798:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384704 00000100:00100000:1.0:1789760793.178852:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774411932721 time 774449560649 state 0x0 ref 1 00000100:00100000:1.0:1789760793.178869:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774449821065 time 774449560649 state 0x1 ref 1 00000100:00100000:1.0:1789760793.178904:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774449821065 time 774494207194 state 0x0 ref 1 00000100:00100000:1.0:1789760793.178915:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507384704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.178962:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23916544, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.178972:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.178987:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.178991:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.178997:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.179011:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.219831:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:1.0:1789760793.219851:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219864:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219872:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219880:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.219884:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219894:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219914:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.219922:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9be4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.220002:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507384704:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41084us (42318us total) trans 4294973159 rc 0/0 00000100:00100000:1.0:1789760793.220012:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5869 00000100:00100000:1.0:1789760793.220022:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774449821065 time 774494207194 state 0x2 ref 0 00000001:00080000:1.0:1789760793.220070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00000100:00100000:2.0:1789760793.225350:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.225835:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384832 00000100:00100000:1.0:1789760793.225845:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384832 00000100:00100000:1.0:1789760793.225904:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774449821065 time 774494207194 state 0x0 ref 1 00000100:00100000:1.0:1789760793.225920:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774494467610 time 774494207194 state 0x1 ref 1 00000100:00100000:0.0:1789760793.226253:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774494467610 time 774541553680 state 0x0 ref 1 00000100:00100000:0.0:1789760793.226269:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384832:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760793.226319:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23920640, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:0.0:1789760793.226328:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226345:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760793.226349:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.226356:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226370:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.308774:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00002000:00010000:1.0:1789760793.308956:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23920640 -> 23924736 00010000:00010000:1.0:1789760793.308971:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.308985:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.308995:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309002:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.309007:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309015:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309034:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309042:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9beb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.309135:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384832:12345-192.168.203.41@tcp:4:dd.0 Request processed in 82867us (83791us total) trans 4294973160 rc 0/0 00000100:00100000:1.0:1789760793.309147:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5870 00000100:00100000:1.0:1789760793.309156:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774494467610 time 774541553680 state 0x2 ref 0 00000100:00100000:1.0:1789760793.312363:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760793.312472:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507384960 00000100:00100000:2.0:1789760793.312482:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507384960 00000100:00100000:2.0:1789760793.312536:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774494467610 time 774541553680 state 0x0 ref 1 00000100:00100000:2.0:1789760793.312551:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774541814096 time 774541553680 state 0x1 ref 1 00000100:00100000:2.0:1789760793.312580:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774541814096 time 774627883471 state 0x0 ref 1 00000100:00100000:2.0:1789760793.312593:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507384960:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760793.312641:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23924736, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:2.0:1789760793.312653:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.312667:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760793.312671:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.312676:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.312945:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.387895:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:1.0:1789760793.387915:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387928:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387938:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387945:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.387950:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387960:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387978:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387990:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9bf2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.388080:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507384960:12345-192.168.203.41@tcp:4:dd.0 Request processed in 75484us (75721us total) trans 4294973161 rc 0/0 00000100:00100000:1.0:1789760793.388090:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5871 00000100:00100000:1.0:1789760793.388099:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774541814096 time 774627883471 state 0x2 ref 0 00000001:00080000:0.0:1789760793.388165:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00000100:00100000:1.0:1789760793.398386:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760793.399518:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385088 00000100:00100000:2.0:1789760793.399529:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385088 00000100:00100000:2.0:1789760793.399588:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774541814096 time 774627883471 state 0x0 ref 1 00000100:00100000:2.0:1789760793.399605:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774628143887 time 774627883471 state 0x1 ref 1 00000100:00100000:2.0:1789760793.399639:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774628143887 time 774714941684 state 0x0 ref 1 00000100:00100000:2.0:1789760793.399652:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507385088:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760793.399736:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23928832, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:2.0:1789760793.399745:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.399762:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760793.399767:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.399773:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.399787:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.442918:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:1.0:1789760793.442939:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442951:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442961:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442968:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.442973:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442982:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.443011:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.443018:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cbc44f4/0x9638aab0164b9bf9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.443120:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507385088:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43467us (44739us total) trans 4294973162 rc 0/0 00000100:00100000:1.0:1789760793.443131:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5872 00000100:00100000:1.0:1789760793.443140:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774628143887 time 774714941684 state 0x2 ref 0 00000001:00080000:2.0:1789760793.446453:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00000100:00100000:2.0:1789760793.448667:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.448969:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385216 00000100:00100000:1.0:1789760793.448980:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385216 00000100:00100000:0.0:1789760793.449184:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774628143887 time 774714941684 state 0x0 ref 1 00000100:00100000:0.0:1789760793.449204:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774715202100 time 774714941684 state 0x1 ref 1 00000100:00100000:0.0:1789760793.449277:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774715202100 time 774764579487 state 0x0 ref 1 00000100:00100000:0.0:1789760793.449290:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507385216:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760793.449350:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23932928, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:0.0:1789760793.449360:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.449375:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760793.449380:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.449387:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.449413:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.493038:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00002000:00010000:1.0:1789760793.493250:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23932928 -> 23937024 00010000:00010000:1.0:1789760793.493266:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493280:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493289:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493297:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.493302:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493327:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493346:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.493355:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9c00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.493643:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507385216:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44348us (44978us total) trans 4294973163 rc 0/0 00000100:00100000:1.0:1789760793.493653:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5873 00000100:00100000:1.0:1789760793.493663:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774715202100 time 774764579487 state 0x2 ref 0 00000100:00100000:3.0:1789760793.498627:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.498940:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385344 00000100:00100000:1.0:1789760793.498951:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385344 00000100:00100000:1.0:1789760793.499008:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774715202100 time 774764579487 state 0x0 ref 1 00000100:00100000:1.0:1789760793.499022:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774764839903 time 774764579487 state 0x1 ref 1 00000100:00100000:1.0:1789760793.499158:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774764839903 time 774814459673 state 0x0 ref 1 00000100:00100000:1.0:1789760793.499173:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385344:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.499222:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23937024, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.499230:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.499248:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.499252:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.499258:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.499271:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.501346:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760793.545041:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:1.0:1789760793.545177:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:1.0:1789760793.545193:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545206:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545215:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545222:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.545227:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545236:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545255:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.545263:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.545639:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385344:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46464us (47187us total) trans 4294973164 rc 0/0 00000100:00100000:1.0:1789760793.545651:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5874 00000100:00100000:1.0:1789760793.545661:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774764839903 time 774814459673 state 0x2 ref 0 00000100:00100000:2.0:1789760793.548775:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.549439:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385472 00000100:00100000:1.0:1789760793.549449:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385472 00000100:00100000:1.0:1789760793.549502:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774764839903 time 774814459673 state 0x0 ref 1 00000100:00100000:1.0:1789760793.549518:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774814720089 time 774814459673 state 0x1 ref 1 00000100:00100000:1.0:1789760793.549618:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774814720089 time 774864920186 state 0x0 ref 1 00000100:00100000:1.0:1789760793.549631:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507385472:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.549680:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23941120, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.549688:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.549724:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.549728:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.549735:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.549754:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.588264:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00002000:00010000:0.0:1789760793.588349:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23941120 -> 23945216 00010000:00010000:0.0:1789760793.588364:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.588377:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.588386:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.588410:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760793.588415:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.588425:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.589261:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.589274:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.589901:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507385472:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40265us (41127us total) trans 4294973165 rc 0/0 00000100:00100000:0.0:1789760793.589912:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5875 00000100:00100000:0.0:1789760793.589922:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774814720089 time 774864920186 state 0x2 ref 0 00000100:00100000:2.0:1789760793.593346:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.593606:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385600 00000100:00100000:1.0:1789760793.593616:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385600 00000100:00100000:1.0:1789760793.593671:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774814720089 time 774864920186 state 0x0 ref 1 00000100:00100000:1.0:1789760793.593687:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774865180602 time 774864920186 state 0x1 ref 1 00000100:00100000:1.0:1789760793.594006:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774865180602 time 774909308220 state 0x0 ref 1 00000100:00100000:1.0:1789760793.594021:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385600:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.594072:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23945216, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.594080:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.594096:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760793.594102:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.594108:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.594122:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.629053:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00002000:00010000:2.0:1789760793.629204:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23945216 -> 23949312 00010000:00010000:2.0:1789760793.629219:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629231:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629240:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629247:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760793.629252:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629266:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629287:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.629295:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9c15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.629429:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385600:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35407us (36086us total) trans 4294973166 rc 0/0 00000100:00100000:2.0:1789760793.629440:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5876 00000100:00100000:2.0:1789760793.629457:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774865180602 time 774909308220 state 0x2 ref 0 00000100:00100000:1.0:1789760793.634824:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760793.635159:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385728 00000100:00100000:2.0:1789760793.635169:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385728 00000100:00100000:2.0:1789760793.635220:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774865180602 time 774909308220 state 0x0 ref 1 00000100:00100000:2.0:1789760793.635235:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774909568636 time 774909308220 state 0x1 ref 1 00000100:00100000:2.0:1789760793.635268:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774909568636 time 774950571604 state 0x0 ref 1 00000100:00100000:2.0:1789760793.635280:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507385728:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760793.635331:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23949312, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:2.0:1789760793.635341:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.635357:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760793.635361:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.635367:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.635379:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.672365:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:2.0:1789760793.672668:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:2.0:1789760793.672684:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672772:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672782:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672789:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760793.672794:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672803:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672822:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.672830:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aafcdbdb/0x9638aab0164b9c1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.673022:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507385728:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37737us (38200us total) trans 4294973167 rc 0/0 00000100:00100000:3.0:1789760793.673032:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5877 00000100:00100000:3.0:1789760793.673042:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774909568636 time 774950571604 state 0x2 ref 0 00000100:00100000:1.0:1789760793.678469:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760793.678615:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385856 00000100:00100000:0.0:1789760793.678625:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385856 00000100:00100000:0.0:1789760793.678682:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774909568636 time 774950571604 state 0x0 ref 1 00000100:00100000:0.0:1789760793.678731:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774950832020 time 774950571604 state 0x1 ref 1 00000100:00100000:0.0:1789760793.678760:0:8090:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774950832020 time 774994062888 state 0x0 ref 1 00000100:00100000:0.0:1789760793.678772:0:8090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507385856:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760793.678821:0:8090:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23953408, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:0.0:1789760793.678829:0:8090:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.678844:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760793.678849:0:8090:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.678856:0:8090:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.678870:0:8090:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760793.714922:0:8090:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:0.0:1789760793.714944:0:8090:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.714957:0:8090:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.714966:0:8090:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.714972:0:8090:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760793.714978:0:8090:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.714987:0:8090:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.715009:0:8090:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.715017:0:8090:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9c23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8090 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.715107:0:8090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507385856:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36332us (36664us total) trans 4294973168 rc 0/0 00000100:00100000:0.0:1789760793.715118:0:8090:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5878 00000100:00100000:0.0:1789760793.715127:0:8090:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774950832020 time 774994062888 state 0x2 ref 0 00000001:00080000:3.0:1789760793.715808:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00000100:00100000:2.0:1789760793.718918:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.718998:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507385984 00000100:00100000:1.0:1789760793.719006:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507385984 00000100:00100000:1.0:1789760793.719053:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774950832020 time 774994062888 state 0x0 ref 1 00000100:00100000:1.0:1789760793.719066:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774994323304 time 774994062888 state 0x1 ref 1 00000100:00100000:1.0:1789760793.719091:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774994323304 time 775034394940 state 0x0 ref 1 00000100:00100000:1.0:1789760793.719101:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385984:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.719143:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23957504, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.719152:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719166:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760793.719171:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.719177:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719191:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.753359:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:3.0:1789760793.753494:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:3.0:1789760793.753509:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753521:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753530:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753537:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dae00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760793.753542:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753552:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753572:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.753580:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5d7e37a/0x9638aab0164b9c2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.753675:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507385984:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34569us (34760us total) trans 4294973169 rc 0/0 00000100:00100000:3.0:1789760793.753685:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5879 00000100:00100000:3.0:1789760793.753725:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774994323304 time 775034394940 state 0x2 ref 0 00000100:00100000:2.0:1789760793.762017:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760793.762092:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386112 00000100:00100000:3.0:1789760793.762103:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386112 00000100:00100000:3.0:1789760793.762154:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 774994323304 time 775034394940 state 0x0 ref 1 00000100:00100000:3.0:1789760793.762169:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775034655356 time 775034394940 state 0x1 ref 1 00000100:00100000:3.0:1789760793.762197:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775034655356 time 775077500648 state 0x0 ref 1 00000100:00100000:3.0:1789760793.762209:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507386112:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760793.762254:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23961600, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:3.0:1789760793.762263:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.762278:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760793.762283:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.762289:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.762304:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.796368:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00002000:00010000:3.0:1789760793.796567:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:3.0:1789760793.796593:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796606:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796616:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796623:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760793.796628:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796637:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796656:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796664:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.797226:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507386112:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35011us (35212us total) trans 4294973170 rc 0/0 00000100:00100000:1.0:1789760793.797238:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5880 00000100:00100000:1.0:1789760793.797248:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775034655356 time 775077500648 state 0x2 ref 0 00000100:00100000:1.0:1789760793.800259:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760793.800378:0:8090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386240 00000100:00100000:0.0:1789760793.800388:0:8090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386240 00000100:00100000:0.0:1789760793.800549:0:8090:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775034655356 time 775077500648 state 0x0 ref 1 00000100:00100000:0.0:1789760793.800566:0:8090:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775077761064 time 775077500648 state 0x1 ref 1 00000100:00100000:3.0:1789760793.800641:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775077761064 time 775115941186 state 0x0 ref 1 00000100:00100000:3.0:1789760793.800655:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507386240:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760793.800723:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23965696, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:3.0:1789760793.800731:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.800746:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760793.800750:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.800756:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.800770:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.843532:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00002000:00010000:2.0:1789760793.843683:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23965696 -> 23969792 00010000:00010000:2.0:1789760793.843727:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843960:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843971:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843979:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760793.843983:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843993:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.844014:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.844028:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9c38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.844236:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507386240:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43578us (43980us total) trans 4294973171 rc 0/0 00000100:00100000:1.0:1789760793.844247:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5881 00000100:00100000:1.0:1789760793.844257:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775077761064 time 775115941186 state 0x2 ref 0 00000100:00100000:2.0:1789760793.848367:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760793.848553:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386368 00000100:00100000:3.0:1789760793.848563:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386368 00000100:00100000:3.0:1789760793.848615:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775077761064 time 775115941186 state 0x0 ref 1 00000100:00100000:3.0:1789760793.848630:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775116201602 time 775115941186 state 0x1 ref 1 00000100:00100000:3.0:1789760793.848660:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775116201602 time 775163963309 state 0x0 ref 1 00000100:00100000:3.0:1789760793.848672:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507386368:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760793.848760:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23969792, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:3.0:1789760793.848770:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.848786:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760793.848790:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.848796:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.848811:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.906354:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23969792 -> 23973888 00010000:00010000:1.0:1789760793.906375:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906388:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906410:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906418:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.906423:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906443:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906463:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.906476:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9c3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.906570:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507386368:12345-192.168.203.41@tcp:4:dd.0 Request processed in 57895us (58209us total) trans 4294973172 rc 0/0 00000100:00100000:1.0:1789760793.906580:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5882 00000100:00100000:1.0:1789760793.906590:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775116201602 time 775163963309 state 0x2 ref 0 00000001:00080000:2.0:1789760793.907254:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00000100:00100000:2.0:1789760793.912762:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.912892:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386496 00000100:00100000:1.0:1789760793.912901:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386496 00000100:00100000:1.0:1789760793.912958:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775116201602 time 775163963309 state 0x0 ref 1 00000100:00100000:1.0:1789760793.912974:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775164223725 time 775163963309 state 0x1 ref 1 00000100:00100000:1.0:1789760793.913008:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775164223725 time 775228310211 state 0x0 ref 1 00000100:00100000:1.0:1789760793.913022:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507386496:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.913071:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23973888, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.913080:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.913097:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760793.913102:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.913108:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.913252:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.942751:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23973888 -> 23977984 00010000:00010000:1.0:1789760793.942769:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942782:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942795:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942802:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760793.942807:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942816:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942833:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.942841:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.942923:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507386496:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29901us (30168us total) trans 4294973173 rc 0/0 00000100:00100000:1.0:1789760793.942932:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5883 00000100:00100000:1.0:1789760793.942942:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775164223725 time 775228310211 state 0x2 ref 0 00000001:00080000:2.0:1789760793.943418:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00000100:00100000:3.0:1789760793.951970:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760793.952181:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386624 00000100:00100000:1.0:1789760793.952191:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386624 00000100:00100000:1.0:1789760793.952247:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775164223725 time 775228310211 state 0x0 ref 1 00000100:00100000:1.0:1789760793.952263:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775228570627 time 775228310211 state 0x1 ref 1 00000100:00100000:1.0:1789760793.952297:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775228570627 time 775267600417 state 0x0 ref 1 00000100:00100000:1.0:1789760793.952310:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507386624:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760793.952356:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23977984, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:1.0:1789760793.952364:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.952379:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760793.952383:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.952389:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.952416:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.995897:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:3.0:1789760793.996025:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:3.0:1789760793.996041:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996054:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996064:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996072:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760793.996078:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996088:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996108:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.996117:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9c4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.996250:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507386624:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43937us (44285us total) trans 4294973174 rc 0/0 00000100:00100000:3.0:1789760793.996261:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5884 00000100:00100000:3.0:1789760793.996271:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775228570627 time 775267600417 state 0x2 ref 0 00000100:00100000:0.0:1789760794.001885:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.001962:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386752 00000100:00100000:3.0:1789760794.001972:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386752 00000100:00100000:3.0:1789760794.002027:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775228570627 time 775267600417 state 0x0 ref 1 00000100:00100000:3.0:1789760794.002042:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775267860833 time 775267600417 state 0x1 ref 1 00000100:00100000:3.0:1789760794.002070:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775267860833 time 775317374212 state 0x0 ref 1 00000100:00100000:3.0:1789760794.002083:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507386752:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.002131:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23982080, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xb807 00010000:00010000:3.0:1789760794.002142:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002158:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.002163:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.002169:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002182:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.039447:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23982080 -> 23986176 00002000:00010000:1.0:1789760794.039457:0:14618:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:1.0:1789760794.039460:0:14618:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:1.0:1789760794.039473:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039486:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039495:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039502:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.039507:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039516:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039535:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.039543:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.039642:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507386752:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37557us (37763us total) trans 4294973175 rc 0/0 00000100:00100000:1.0:1789760794.039652:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5885 00000100:00100000:1.0:1789760794.039662:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775267860833 time 775317374212 state 0x2 ref 0 00000001:00080000:2.0:1789760794.039831:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00000100:00100000:3.0:1789760794.042388:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.043075:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507386880 00000100:00100000:1.0:1789760794.043085:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507386880 00000100:00100000:1.0:1789760794.043137:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775267860833 time 775317374212 state 0x0 ref 1 00000100:00100000:1.0:1789760794.043152:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775317634628 time 775317374212 state 0x1 ref 1 00000100:00100000:1.0:1789760794.043215:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775317634628 time 775358517318 state 0x0 ref 1 00000100:00100000:1.0:1789760794.043227:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507386880:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.043273:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23986176, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.043281:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.043297:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760794.043302:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.043307:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.043321:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.073661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:1.0:1789760794.073929:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23986176 -> 23990272 00010000:00010000:1.0:1789760794.074137:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074151:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074229:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074237:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.074242:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074251:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074270:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.074278:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9c5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.074413:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507386880:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31183us (32028us total) trans 4294973176 rc 0/0 00000100:00100000:1.0:1789760794.074428:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5886 00000100:00100000:1.0:1789760794.074446:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775317634628 time 775358517318 state 0x2 ref 0 00000100:00100000:2.0:1789760794.078957:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.079035:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387008 00000100:00100000:3.0:1789760794.079045:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387008 00000100:00100000:3.0:1789760794.079099:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775317634628 time 775358517318 state 0x0 ref 1 00000100:00100000:3.0:1789760794.079114:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775358777734 time 775358517318 state 0x1 ref 1 00000100:00100000:3.0:1789760794.079173:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775358777734 time 775394476679 state 0x0 ref 1 00000100:00100000:3.0:1789760794.079188:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507387008:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.079233:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23990272, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.079243:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.079259:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.079264:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.079270:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.079283:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.117064:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00002000:00010000:3.0:1789760794.117165:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:3.0:1789760794.117179:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117192:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117201:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117208:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.117213:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117222:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117241:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.117248:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.117336:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507387008:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38147us (38383us total) trans 4294973177 rc 0/0 00000100:00100000:3.0:1789760794.117345:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5887 00000100:00100000:3.0:1789760794.117354:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775358777734 time 775394476679 state 0x2 ref 0 00000100:00100000:3.0:1789760794.121034:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.121179:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387136 00000100:00100000:1.0:1789760794.121190:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387136 00000100:00100000:1.0:1789760794.121243:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775358777734 time 775394476679 state 0x0 ref 1 00000100:00100000:1.0:1789760794.121257:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775394737095 time 775394476679 state 0x1 ref 1 00000100:00100000:1.0:1789760794.121290:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775394737095 time 775436592633 state 0x0 ref 1 00000100:00100000:1.0:1789760794.121302:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507387136:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.121347:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23994368, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.121357:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.121375:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760794.121380:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.121386:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.121411:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.150453:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00002000:00010000:1.0:1789760794.150677:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:1.0:1789760794.150759:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150773:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150782:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150789:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.150794:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150803:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150823:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.150829:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.151029:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507387136:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29724us (30000us total) trans 4294973178 rc 0/0 00000100:00100000:1.0:1789760794.151040:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5888 00000100:00100000:1.0:1789760794.151049:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775394737095 time 775436592633 state 0x2 ref 0 00000100:00100000:1.0:1789760794.155406:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760794.155557:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387264 00000100:00100000:2.0:1789760794.155568:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387264 00000100:00100000:2.0:1789760794.155620:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775394737095 time 775436592633 state 0x0 ref 1 00000100:00100000:2.0:1789760794.155636:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775436853049 time 775436592633 state 0x1 ref 1 00000100:00100000:2.0:1789760794.155729:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775436853049 time 775471031484 state 0x0 ref 1 00000100:00100000:2.0:1789760794.155742:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507387264:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760794.155789:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23998464, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:2.0:1789760794.155799:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.155816:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760794.155821:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.155827:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.155852:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.186683:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00002000:00010000:3.0:1789760794.186929:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:3.0:1789760794.186945:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186959:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186968:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186976:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.186980:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186989:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.187009:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.187022:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9c70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.187144:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507387264:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31400us (31755us total) trans 4294973179 rc 0/0 00000100:00100000:3.0:1789760794.187216:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5889 00000100:00100000:3.0:1789760794.187226:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775436853049 time 775471031484 state 0x2 ref 0 00000100:00100000:2.0:1789760794.190978:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.191097:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387392 00000100:00100000:3.0:1789760794.191106:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387392 00000100:00100000:3.0:1789760794.191219:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775436853049 time 775471031484 state 0x0 ref 1 00000100:00100000:3.0:1789760794.191236:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775471291900 time 775471031484 state 0x1 ref 1 00000100:00100000:3.0:1789760794.191364:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775471291900 time 775506666481 state 0x0 ref 1 00000100:00100000:3.0:1789760794.191377:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507387392:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.191498:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24002560, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.191507:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.191524:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.191529:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.191535:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.191554:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.250182:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00002000:00010000:3.0:1789760794.250344:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:3.0:1789760794.250361:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250374:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250384:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250392:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.250408:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250416:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250635:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.250644:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.250818:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507387392:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59438us (59846us total) trans 4294973180 rc 0/0 00000100:00100000:3.0:1789760794.250829:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5890 00000100:00100000:3.0:1789760794.250848:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775471291900 time 775506666481 state 0x2 ref 0 00000100:00100000:3.0:1789760794.256851:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.256917:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387520 00000100:00100000:1.0:1789760794.256928:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387520 00000100:00100000:1.0:1789760794.257080:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775471291900 time 775506666481 state 0x0 ref 1 00000100:00100000:1.0:1789760794.257095:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775506926897 time 775506666481 state 0x1 ref 1 00000100:00100000:1.0:1789760794.257475:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775506926897 time 775572775567 state 0x0 ref 1 00000100:00100000:1.0:1789760794.257491:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507387520:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.257544:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24006656, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.257554:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.257571:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760794.257577:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.257583:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.257599:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.307445:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00002000:00010000:3.0:1789760794.307633:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:3.0:1789760794.307650:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307663:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307672:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307679:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.307683:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307708:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307729:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.307741:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9c7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.307836:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507387520:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50343us (50999us total) trans 4294973181 rc 0/0 00000100:00100000:3.0:1789760794.308872:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5891 00000100:00100000:3.0:1789760794.308883:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775506926897 time 775572775567 state 0x2 ref 0 00000100:00100000:0.0:1789760794.312236:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.312371:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387648 00000100:00100000:3.0:1789760794.312381:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387648 00000100:00100000:3.0:1789760794.312449:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775506926897 time 775572775567 state 0x0 ref 1 00000100:00100000:3.0:1789760794.312536:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775573035983 time 775572775567 state 0x1 ref 1 00000100:00100000:3.0:1789760794.312568:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775573035983 time 775627872467 state 0x0 ref 1 00000100:00100000:3.0:1789760794.312580:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507387648:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.312624:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24010752, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.312634:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.312650:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.312655:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.312661:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.312681:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.326944:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.326976:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.327000:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.327022:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.327041:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.327066:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387904 00000100:00100000:3.0:1789760794.327075:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387904 00000100:00100000:3.0:1789760794.327129:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388160 00000100:00100000:3.0:1789760794.327132:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388160 00000100:00100000:3.0:1789760794.327149:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388032 00000100:00100000:3.0:1789760794.327162:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388032 00000100:00100000:3.0:1789760794.327176:0:7721:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388288 00000100:00100000:3.0:1789760794.327179:0:7721:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388288 00000100:00100000:3.0:1789760794.327194:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 988 00000100:00100000:3.0:1789760794.327203:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507387904:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760794.327273:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105823480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+8:2417:x1876699507387904:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 71us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327281:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 988 00000100:00100000:3.0:1789760794.327296:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 989 00000100:00100000:3.0:1789760794.327303:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507388160:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760794.327328:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507388160:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 25us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327337:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 989 00000100:00100000:3.0:1789760794.327349:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 990 00000100:00100000:3.0:1789760794.327355:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388032:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760794.327377:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388032:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 22us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327384:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 990 00000100:00100000:3.0:1789760794.327406:0:7721:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 991 00000100:00100000:3.0:1789760794.327413:0:7721:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507388288:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760794.327447:0:7721:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c105821880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0668622-7dd7-45f4-a020-184f7e88cc06+5:2417:x1876699507388288:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 35us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327453:0:7721:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 991 00000100:00100000:3.0:1789760794.327646:0:6434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507387776 00000100:00100000:3.0:1789760794.327651:0:6434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507387776 00000100:00100000:3.0:1789760794.327673:0:6434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.41@tcp, seq: 488 00000100:00100000:3.0:1789760794.327679:0:6434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c203c86450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507387776:12345-192.168.203.41@tcp:400:kworker.0 00000100:00100000:3.0:1789760794.327730:0:6434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c203c86450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7c97d591-104c-4fe1-aa51-5b4983d2eb89+13:2417:x1876699507387776:12345-192.168.203.41@tcp:400:kworker.0 Request processed in 51us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327736:0:6434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.41@tcp, seq: 488 00000100:00080000:2.0:1789760794.328386:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760794.328390:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:3.0:1789760794.361881:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:3.0:1789760794.361901:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361913:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361927:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361934:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.361938:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361946:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361962:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.361969:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.362057:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507387648:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49474us (49826us total) trans 4294973182 rc 0/0 00000100:00100000:3.0:1789760794.362066:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5892 00000100:00100000:3.0:1789760794.362076:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775573035983 time 775627872467 state 0x2 ref 0 00000001:00080000:0.0:1789760794.363102:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00000100:00100000:3.0:1789760794.366287:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760794.366644:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388416 00000100:00100000:0.0:1789760794.366654:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388416 00000100:00100000:0.0:1789760794.366769:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775573035983 time 775627872467 state 0x0 ref 1 00000100:00100000:0.0:1789760794.366791:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775628132883 time 775627872467 state 0x1 ref 1 00000100:00100000:0.0:1789760794.366815:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775628132883 time 775682118832 state 0x0 ref 1 00000100:00100000:0.0:1789760794.366843:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388416:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760794.366926:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24014848, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:0.0:1789760794.366935:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.366964:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760794.366969:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.366975:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.366995:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.403918:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00002000:00010000:3.0:1789760794.404057:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:3.0:1789760794.404072:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404092:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404100:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404107:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.404112:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404129:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404148:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.404161:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9c8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.404245:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388416:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37409us (37965us total) trans 4294973183 rc 0/0 00000100:00100000:3.0:1789760794.404255:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5893 00000100:00100000:3.0:1789760794.404264:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775628132883 time 775682118832 state 0x2 ref 0 00000100:00100000:2.0:1789760794.408729:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.408798:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388544 00000100:00100000:3.0:1789760794.408809:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388544 00000100:00100000:3.0:1789760794.408868:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775628132883 time 775682118832 state 0x0 ref 1 00000100:00100000:3.0:1789760794.408883:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775682379248 time 775682118832 state 0x1 ref 1 00000100:00100000:3.0:1789760794.408909:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775682379248 time 775724212872 state 0x0 ref 1 00000100:00100000:3.0:1789760794.408919:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388544:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.408962:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24018944, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.408971:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.408985:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.408989:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.408996:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.409009:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.446358:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:3.0:1789760794.446566:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24018944 -> 24023040 00010000:00010000:3.0:1789760794.446581:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446594:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446609:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446616:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.446621:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446629:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446648:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.446656:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9c93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.446808:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388544:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37885us (38110us total) trans 4294973184 rc 0/0 00000100:00100000:3.0:1789760794.446817:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5894 00000100:00100000:3.0:1789760794.446829:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775682379248 time 775724212872 state 0x2 ref 0 00000100:00100000:2.0:1789760794.451319:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.451410:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388672 00000100:00100000:3.0:1789760794.451421:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388672 00000100:00100000:3.0:1789760794.451553:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775682379248 time 775724212872 state 0x0 ref 1 00000100:00100000:3.0:1789760794.451569:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775724473288 time 775724212872 state 0x1 ref 1 00000100:00100000:3.0:1789760794.451598:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775724473288 time 775766901089 state 0x0 ref 1 00000100:00100000:3.0:1789760794.451610:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388672:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.451662:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24023040, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.451671:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.451687:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.451726:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.451733:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.451747:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.476530:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00002000:00010000:3.0:1789760794.476623:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:3.0:1789760794.476638:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476650:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476658:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476665:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.476670:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476679:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476717:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.476725:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9c9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.476816:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388672:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25204us (25501us total) trans 4294973185 rc 0/0 00000100:00100000:3.0:1789760794.476827:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5895 00000100:00100000:3.0:1789760794.479086:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775724473288 time 775766901089 state 0x2 ref 0 00000100:00100000:1.0:1789760794.480270:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.480341:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388800 00000100:00100000:3.0:1789760794.480351:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388800 00000100:00100000:3.0:1789760794.480589:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775724473288 time 775766901089 state 0x0 ref 1 00000100:00100000:3.0:1789760794.480603:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775767161505 time 775766901089 state 0x1 ref 1 00000100:00100000:3.0:1789760794.480636:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775767161505 time 775795939044 state 0x0 ref 1 00000100:00100000:3.0:1789760794.480647:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388800:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.480722:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24027136, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.480732:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.480748:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.480753:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.480758:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.480772:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.504785:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00002000:00010000:3.0:1789760794.504918:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24027136 -> 24031232 00010000:00010000:3.0:1789760794.504933:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.504946:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.504957:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.504964:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.504968:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.504976:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.505001:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.505008:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9ca1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.505097:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388800:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24446us (24829us total) trans 4294973186 rc 0/0 00000100:00100000:3.0:1789760794.505106:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5896 00000100:00100000:3.0:1789760794.505116:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775767161505 time 775795939044 state 0x2 ref 0 00000100:00100000:3.0:1789760794.508466:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.508799:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507388928 00000100:00100000:1.0:1789760794.508811:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507388928 00000100:00100000:1.0:1789760794.508932:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775767161505 time 775795939044 state 0x0 ref 1 00000100:00100000:1.0:1789760794.508948:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775796199460 time 775795939044 state 0x1 ref 1 00000100:00100000:1.0:1789760794.509017:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775796199460 time 775824320019 state 0x0 ref 1 00000100:00100000:1.0:1789760794.509030:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414d4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388928:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.509081:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24031232, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.509090:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.509106:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760794.509110:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.509117:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.509130:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.524712:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760794.535538:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:0.0:1789760794.535716:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:0.0:1789760794.535732:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535751:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535759:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535767:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.535772:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535781:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535801:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.535810:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9ca8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.535902:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414d4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507388928:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26869us (27440us total) trans 4294973187 rc 0/0 00000100:00100000:0.0:1789760794.535913:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5897 00000100:00100000:0.0:1789760794.535922:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775796199460 time 775824320019 state 0x2 ref 0 00000100:00100000:1.0:1789760794.540420:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.540558:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389056 00000100:00100000:3.0:1789760794.540567:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389056 00000100:00100000:3.0:1789760794.540623:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775796199460 time 775824320019 state 0x0 ref 1 00000100:00100000:3.0:1789760794.540640:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775824580435 time 775824320019 state 0x1 ref 1 00000100:00100000:3.0:1789760794.540671:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775824580435 time 775855974062 state 0x0 ref 1 00000100:00100000:3.0:1789760794.540683:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507389056:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.540749:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24035328, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.540759:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.540781:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.540785:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.540792:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.540805:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.573187:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:0.0:1789760794.573275:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:0.0:1789760794.573290:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573302:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573311:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573319:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.573323:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573333:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573351:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.573359:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9caf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.573633:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507389056:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32945us (33216us total) trans 4294973188 rc 0/0 00000100:00100000:0.0:1789760794.573644:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5898 00000100:00100000:0.0:1789760794.573654:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775824580435 time 775855974062 state 0x2 ref 0 00000100:00100000:3.0:1789760794.578749:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760794.578890:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389184 00000100:00100000:0.0:1789760794.578902:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389184 00000100:00100000:0.0:1789760794.578957:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775824580435 time 775855974062 state 0x0 ref 1 00000100:00100000:0.0:1789760794.578973:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775856234478 time 775855974062 state 0x1 ref 1 00000100:00100000:0.0:1789760794.578996:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775856234478 time 775894298998 state 0x0 ref 1 00000100:00100000:0.0:1789760794.579009:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389184:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760794.579059:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24039424, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:0.0:1789760794.579067:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.579084:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760794.579088:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.579095:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.579109:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.634514:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24039424 -> 24043520 00010000:00010000:0.0:1789760794.634546:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634559:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634569:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634576:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.634580:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634589:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634608:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.634616:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9cb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.634726:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507389184:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55685us (55954us total) trans 4294973189 rc 0/0 00000100:00100000:0.0:1789760794.634736:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5899 00000100:00100000:0.0:1789760794.634746:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775856234478 time 775894298998 state 0x2 ref 0 00000001:00080000:0.0:1789760794.634779:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00000100:00100000:3.0:1789760794.637682:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760794.637859:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389312 00000100:00100000:0.0:1789760794.637869:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389312 00000100:00100000:0.0:1789760794.637921:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775856234478 time 775894298998 state 0x0 ref 1 00000100:00100000:0.0:1789760794.637938:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775894559414 time 775894298998 state 0x1 ref 1 00000100:00100000:0.0:1789760794.637969:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775894559414 time 775953272508 state 0x0 ref 1 00000100:00100000:0.0:1789760794.637981:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389312:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760794.638041:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24043520, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:0.0:1789760794.638050:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.638066:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760794.638070:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.638077:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.638092:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.671575:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:0.0:1789760794.671595:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671609:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671617:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671628:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.671632:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671640:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671816:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.671825:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.671913:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507389312:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33928us (34232us total) trans 4294973190 rc 0/0 00000100:00100000:0.0:1789760794.671923:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5900 00000100:00100000:0.0:1789760794.671934:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775894559414 time 775953272508 state 0x2 ref 0 00000001:00080000:1.0:1789760794.672011:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00000100:00100000:0.0:1789760794.682803:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.682883:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389440 00000100:00100000:3.0:1789760794.682893:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389440 00000100:00100000:3.0:1789760794.682942:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775894559414 time 775953272508 state 0x0 ref 1 00000100:00100000:3.0:1789760794.682956:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775953532924 time 775953272508 state 0x1 ref 1 00000100:00100000:3.0:1789760794.682986:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775953532924 time 775998289691 state 0x0 ref 1 00000100:00100000:3.0:1789760794.682997:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389440:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.683041:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24047616, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.683050:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.683064:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760794.683069:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.683075:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.683090:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.721380:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:3.0:1789760794.721534:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:3.0:1789760794.721550:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721563:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721572:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721579:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.721584:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721592:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721612:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.721620:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.721831:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389440:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38830us (39032us total) trans 4294973191 rc 0/0 00000100:00100000:3.0:1789760794.721842:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5901 00000100:00100000:3.0:1789760794.721851:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775953532924 time 775998289691 state 0x2 ref 0 00000100:00100000:3.0:1789760794.726425:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760794.726580:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389568 00000100:00100000:0.0:1789760794.726590:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389568 00000100:00100000:0.0:1789760794.726641:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775953532924 time 775998289691 state 0x0 ref 1 00000100:00100000:0.0:1789760794.726862:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775998550107 time 775998289691 state 0x1 ref 1 00000100:00100000:1.0:1789760794.726993:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775998550107 time 776042293784 state 0x0 ref 1 00000100:00100000:1.0:1789760794.727007:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507389568:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.727054:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24051712, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.727063:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.727077:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760794.727082:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.727088:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.727102:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.752596:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:1.0:1789760794.752617:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752632:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752640:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752647:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.752652:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752673:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752713:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.752722:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9ccb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.752818:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507389568:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25808us (26396us total) trans 4294973192 rc 0/0 00000100:00100000:1.0:1789760794.752827:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5902 00000100:00100000:1.0:1789760794.752837:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775998550107 time 776042293784 state 0x2 ref 0 00000001:00080000:0.0:1789760794.753329:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00000100:00100000:1.0:1789760794.756834:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760794.756921:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389696 00000100:00100000:0.0:1789760794.756930:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389696 00000100:00100000:0.0:1789760794.756983:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 775998550107 time 776042293784 state 0x0 ref 1 00000100:00100000:0.0:1789760794.756997:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776042554200 time 776042293784 state 0x1 ref 1 00000100:00100000:0.0:1789760794.757058:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776042554200 time 776072360774 state 0x0 ref 1 00000100:00100000:0.0:1789760794.757071:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389696:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760794.757118:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24055808, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:0.0:1789760794.757127:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.757143:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760794.757147:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.757154:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.757168:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.786286:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00002000:00010000:0.0:1789760794.786499:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:0.0:1789760794.786514:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786532:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786541:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786547:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.786552:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786625:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786644:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.786651:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9cd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.787069:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389696:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29994us (30237us total) trans 4294973193 rc 0/0 00000100:00100000:1.0:1789760794.787080:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5903 00000100:00100000:1.0:1789760794.787089:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776042554200 time 776072360774 state 0x2 ref 0 00000100:00100000:2.0:1789760794.793810:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.793897:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389824 00000100:00100000:3.0:1789760794.793907:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389824 00000100:00100000:3.0:1789760794.793959:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776042554200 time 776072360774 state 0x0 ref 1 00000100:00100000:3.0:1789760794.793973:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776072621190 time 776072360774 state 0x1 ref 1 00000100:00100000:3.0:1789760794.794002:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776072621190 time 776109305904 state 0x0 ref 1 00000100:00100000:3.0:1789760794.794014:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389824:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.794063:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24059904, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.794073:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.794088:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760794.794092:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.794098:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.794112:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.837212:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00002000:00010000:3.0:1789760794.838312:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24059904 -> 24064000 00010000:00010000:3.0:1789760794.838332:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838345:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838354:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838360:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.838365:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838373:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838401:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.838409:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9cd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.838526:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507389824:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44509us (44720us total) trans 4294973194 rc 0/0 00000100:00100000:3.0:1789760794.838540:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5904 00000100:00100000:3.0:1789760794.838550:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776072621190 time 776109305904 state 0x2 ref 0 00000100:00100000:3.0:1789760794.842942:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.843072:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507389952 00000100:00100000:1.0:1789760794.843083:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507389952 00000100:00100000:1.0:1789760794.843138:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776072621190 time 776109305904 state 0x0 ref 1 00000100:00100000:1.0:1789760794.843153:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776109566320 time 776109305904 state 0x1 ref 1 00000100:00100000:1.0:1789760794.843178:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776109566320 time 776158481160 state 0x0 ref 1 00000100:00100000:1.0:1789760794.843194:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507389952:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.843246:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24064000, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.843255:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.843270:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760794.843274:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.843281:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.843295:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.845310:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538297856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845374:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845444:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538298368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845470:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845493:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c23473b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538298880:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845513:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845537:0:3463:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789760794.845543:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108022300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538297472:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760794.845563:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845579:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538297984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845600:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845615:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234738700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538298496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845641:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845658:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c79c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538299008:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845677:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845721:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108023b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538297600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845748:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845768:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538298112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845791:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845807:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234739c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538298624:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845838:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845853:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107c78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538299136:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845869:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845885:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1789760794.845890:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c108020a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538297728:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845906:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845918:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c107e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538298240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845934:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845945:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234738e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538298752:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.845961:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.845984:0:9478:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297472 00000100:00100000:2.0:1789760794.845990:0:9478:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297472 00000100:00100000:2.0:1789760794.846033:0:9478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789760794.846043:0:9478:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538297472:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760794.846079:0:9478:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760794.846117:0:9478:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538297472:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (553us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846125:0:9478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789760794.846144:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298496 00000100:00100000:2.0:1789760794.846147:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298496 00000100:00100000:2.0:1789760794.846171:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299008 00000100:00100000:2.0:1789760794.846173:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299008 00000100:00100000:2.0:1789760794.846182:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298624 00000100:00100000:2.0:1789760794.846184:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298624 00000100:00100000:2.0:1789760794.846195:0:8084:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299136 00000100:00100000:2.0:1789760794.846197:0:8084:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299136 00000100:00100000:2.0:1789760794.846209:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:2.0:1789760794.846214:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538298496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846242:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538298496:12345-0@lo:400:kworker.0 Request processed in 28us (601us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846248:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:2.0:1789760794.846257:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:2.0:1789760794.846261:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538299008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846279:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538299008:12345-0@lo:400:kworker.0 Request processed in 18us (602us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846285:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:2.0:1789760794.846292:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:2.0:1789760794.846297:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538298624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846318:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538298624:12345-0@lo:400:kworker.0 Request processed in 22us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846324:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:2.0:1789760794.846332:0:8084:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:2.0:1789760794.846337:0:8084:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538299136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846357:0:8084:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538299136:12345-0@lo:400:kworker.0 Request processed in 21us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846363:0:8084:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:2.0:1789760794.846378:0:6433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297600 00000100:00100000:2.0:1789760794.846380:0:6433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297600 00000100:00100000:2.0:1789760794.846414:0:6433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:2.0:1789760794.846421:0:6433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d6bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538297600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846446:0:6433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d6bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a12369fc-e31f-4eec-83dc-16c04178fd0b+24:3461:x1876699538297600:12345-0@lo:400:kworker.0 Request processed in 26us (697us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846451:0:6433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:2.0:1789760794.846476:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297856 00000100:00100000:2.0:1789760794.846480:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297856 00000100:00100000:2.0:1789760794.846497:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298112 00000100:00100000:2.0:1789760794.846500:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298112 00000100:00100000:2.0:1789760794.846511:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:2.0:1789760794.846517:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538297856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846547:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3463:x1876699538297856:12345-0@lo:400:kworker.0 Request processed in 30us (1172us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846553:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:2.0:1789760794.846564:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789760794.846570:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538298112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846594:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538298112:12345-0@lo:400:kworker.0 Request processed in 25us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.846599:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00080000:2.0:1789760794.846621:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760794.846625:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760794.846653:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108023b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a12369fc-e31f-4eec-83dc-16c04178fd0b:3461:1876699538297600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846665:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538298112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846674:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234739c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538298624:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.846682:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538299136:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.847456:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298368 00000100:00100000:0.0:1789760794.847462:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298368 00000100:00100000:2.0:1789760794.847486:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760794.847498:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298880 00000100:00100000:2.0:1789760794.847502:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c227868e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538299264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760794.847503:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298880 00000100:00100000:0.0:1789760794.847522:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297984 00000100:00100000:2.0:1789760794.847546:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789760794.847555:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760794.847584:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299264 00000100:00100000:0.0:1789760794.847586:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297984 00000100:00100000:2.0:1789760794.847589:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299264 00000100:00100000:0.0:1789760794.847611:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538297728 00000100:00100000:0.0:1789760794.847616:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538297728 00000100:00100000:0.0:1789760794.847631:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298240 00000100:00100000:0.0:1789760794.847633:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298240 00000100:00100000:2.0:1789760794.847648:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:0.0:1789760794.847659:0:6441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538298752 00000100:00100000:2.0:1789760794.847659:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538299264:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760794.847664:0:6441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538298752 00002000:00000024:2.0:1789760794.847679:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760794.847685:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 992 00000100:00100000:2.0:1789760794.847735:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3461:x1876699538299264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (189us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.847743:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:0.0:1789760794.847788:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538298368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.847789:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538297472:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760794.847807:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234738700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538298496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.847832:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107c79c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538299008:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.847846:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108022a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538297856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.847865:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3463:1876699538298368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.847897:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c227868e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3461:1876699538299264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760794.847971:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3463:x1876699538298368:12345-0@lo:400:kworker.0 Request processed in 277us (2499us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.847977:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 992 00000100:00100000:0.0:1789760794.847989:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 993 00000100:00100000:0.0:1789760794.847995:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538298880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848028:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3463:x1876699538298880:12345-0@lo:400:kworker.0 Request processed in 34us (2514us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.848033:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 993 00000100:00100000:0.0:1789760794.848042:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 994 00000100:00100000:0.0:1789760794.848047:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538297984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848073:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c10802ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3463:x1876699538297984:12345-0@lo:400:kworker.0 Request processed in 26us (2473us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.848077:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 994 00000100:00100000:0.0:1789760794.848086:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 995 00000100:00100000:0.0:1789760794.848090:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538297728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848115:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3461:x1876699538297728:12345-0@lo:400:kworker.0 Request processed in 25us (2208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.848120:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 995 00000100:00100000:0.0:1789760794.848130:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 996 00000100:00100000:0.0:1789760794.848136:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538298240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848157:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22353fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3461:x1876699538298240:12345-0@lo:400:kworker.0 Request processed in 22us (2223us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.848163:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 996 00000100:00100000:0.0:1789760794.848171:0:6441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 997 00000100:00100000:0.0:1789760794.848175:0:6441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538298752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848195:0:6441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3461:x1876699538298752:12345-0@lo:400:kworker.0 Request processed in 20us (2234us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.848200:0:6441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 997 00000100:00080000:0.0:1789760794.848220:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760794.848247:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108020a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3461:1876699538297728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848259:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c107e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3461:1876699538298240:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.848270:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234738e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3461:1876699538298752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.850183:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c108021180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3463:1876699538297984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.850201:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c23473b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3463:1876699538298880:0@lo:400:kworker.0 00000001:00080000:3.0:1789760794.877622:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:1.0:1789760794.877749:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24064000 -> 24068096 00010000:00010000:1.0:1789760794.877765:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877779:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877788:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877802:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.877807:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877816:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877835:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.877843:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032390679/0x9638aab0164b9ce0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.877959:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507389952:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34756us (35014us total) trans 4294973195 rc 0/0 00000100:00100000:1.0:1789760794.877969:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5905 00000100:00100000:1.0:1789760794.877980:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776109566320 time 776158481160 state 0x2 ref 0 00000100:00100000:1.0:1789760794.882353:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760794.882515:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390080 00000100:00100000:2.0:1789760794.882525:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390080 00000100:00100000:2.0:1789760794.882579:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776109566320 time 776158481160 state 0x0 ref 1 00000100:00100000:2.0:1789760794.882594:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776158741576 time 776158481160 state 0x1 ref 1 00000100:00100000:2.0:1789760794.882738:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776158741576 time 776197967609 state 0x0 ref 1 00000100:00100000:2.0:1789760794.882751:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507390080:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760794.882798:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24068096, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:2.0:1789760794.882807:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.882822:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760794.882827:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.882961:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.882979:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.909152:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00002000:00010000:1.0:1789760794.909251:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24068096 -> 24072192 00010000:00010000:1.0:1789760794.909266:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909280:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909294:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909302:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760794.909307:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909316:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909337:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.909344:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9ce7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.909489:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507390080:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26736us (27141us total) trans 4294973196 rc 0/0 00000100:00100000:1.0:1789760794.909499:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5906 00000100:00100000:1.0:1789760794.909510:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776158741576 time 776197967609 state 0x2 ref 0 00000100:00100000:0.0:1789760794.913008:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760794.913090:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390208 00000100:00100000:3.0:1789760794.913099:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390208 00000100:00100000:3.0:1789760794.913150:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776158741576 time 776197967609 state 0x0 ref 1 00000100:00100000:3.0:1789760794.913165:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776198228025 time 776197967609 state 0x1 ref 1 00000100:00100000:3.0:1789760794.913197:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776198228025 time 776228499902 state 0x0 ref 1 00000100:00100000:3.0:1789760794.913210:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507390208:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760794.913256:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24072192, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760794.913265:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.913280:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760794.913285:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.913291:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.913305:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.932852:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:0.0:1789760794.933012:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:0.0:1789760794.933027:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933040:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933048:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933055:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760794.933061:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933070:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933089:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933096:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9cee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.933185:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22786ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507390208:12345-192.168.203.41@tcp:4:dd.0 Request processed in 19973us (20181us total) trans 4294973197 rc 0/0 00000100:00100000:0.0:1789760794.933196:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5907 00000100:00100000:0.0:1789760794.933205:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776198228025 time 776228499902 state 0x2 ref 0 00000100:00100000:3.0:1789760794.936124:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760794.936273:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390336 00000100:00100000:1.0:1789760794.936282:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390336 00000100:00100000:1.0:1789760794.936333:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776198228025 time 776228499902 state 0x0 ref 1 00000100:00100000:1.0:1789760794.936348:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776228760318 time 776228499902 state 0x1 ref 1 00000100:00100000:1.0:1789760794.936381:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776228760318 time 776251683829 state 0x0 ref 1 00000100:00100000:1.0:1789760794.936407:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507390336:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760794.936465:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24076288, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:1.0:1789760794.936473:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.936488:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760794.936492:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.936499:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.936513:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.954581:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:3.0:1789760794.954767:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:3.0:1789760794.954783:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954795:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954804:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954812:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.954817:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954826:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954844:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.954851:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9cf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.954941:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507390336:12345-192.168.203.41@tcp:4:dd.0 Request processed in 18533us (18820us total) trans 4294973198 rc 0/0 00000100:00100000:3.0:1789760794.954952:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5908 00000100:00100000:3.0:1789760794.954961:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776228760318 time 776251683829 state 0x2 ref 0 00000100:00100000:1.0:1789760794.958325:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760794.958415:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390464 00000100:00100000:2.0:1789760794.958424:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390464 00000100:00100000:2.0:1789760794.958480:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776228760318 time 776251683829 state 0x0 ref 1 00000100:00100000:2.0:1789760794.958496:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776251944245 time 776251683829 state 0x1 ref 1 00000100:00100000:2.0:1789760794.958527:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776251944245 time 776273830759 state 0x0 ref 1 00000100:00100000:2.0:1789760794.958543:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507390464:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760794.958595:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24080384, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:2.0:1789760794.958602:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.958618:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760794.958622:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.958627:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.958640:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.995070:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00002000:00010000:3.0:1789760794.995240:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:3.0:1789760794.995256:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995269:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995278:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995285:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760794.995290:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995300:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995320:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.995335:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9cfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.995445:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507390464:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36899us (37123us total) trans 4294973199 rc 0/0 00000100:00100000:3.0:1789760794.995456:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5909 00000100:00100000:3.0:1789760794.995465:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776251944245 time 776273830759 state 0x2 ref 0 00000100:00100000:0.0:1789760795.000880:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.000974:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390592 00000100:00100000:3.0:1789760795.001002:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390592 00000100:00080000:0.0:1789760795.001086:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760795.001111:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776251944245 time 776273830759 state 0x0 ref 1 00000100:00100000:3.0:1789760795.001131:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776274091175 time 776273830759 state 0x1 ref 1 00000100:00100000:3.0:1789760795.001165:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776274091175 time 776316466898 state 0x0 ref 1 00000100:00100000:3.0:1789760795.001182:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390592:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.001345:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24084480, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xb807 00010000:00010000:3.0:1789760795.001361:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.001386:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.001401:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.001409:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.001506:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.034556:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00002000:00010000:3.0:1789760795.038034:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24084480 -> 24088576 00002000:00010000:3.0:1789760795.038044:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:3.0:1789760795.038047:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:3.0:1789760795.038060:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038073:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038082:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038088:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.038093:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038102:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038123:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.038137:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.038238:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390592:12345-192.168.203.41@tcp:4:dd.0 Request processed in 37055us (37363us total) trans 4294973200 rc 0/0 00000100:00100000:3.0:1789760795.038248:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5910 00000100:00100000:3.0:1789760795.038257:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776274091175 time 776316466898 state 0x2 ref 0 00000100:00100000:2.0:1789760795.044071:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.044157:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390720 00000100:00100000:3.0:1789760795.044167:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390720 00000100:00100000:3.0:1789760795.044215:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776274091175 time 776316466898 state 0x0 ref 1 00000100:00100000:3.0:1789760795.044230:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776316727314 time 776316466898 state 0x1 ref 1 00000100:00100000:3.0:1789760795.044264:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776316727314 time 776359567435 state 0x0 ref 1 00000100:00100000:3.0:1789760795.044277:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390720:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.044325:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24088576, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:3.0:1789760795.044334:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.044350:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.044355:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.044360:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.044379:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.089110:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00002000:00010000:2.0:1789760795.092604:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:2.0:1789760795.092625:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092638:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092648:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092655:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.092660:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092669:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092689:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.092712:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9d0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.092806:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390720:12345-192.168.203.41@tcp:4:dd.0 Request processed in 48525us (48739us total) trans 4294973201 rc 0/0 00000100:00100000:2.0:1789760795.092817:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5911 00000100:00100000:2.0:1789760795.092826:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776316727314 time 776359567435 state 0x2 ref 0 00000100:00100000:1.0:1789760795.101398:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.101530:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390848 00000100:00100000:3.0:1789760795.101540:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390848 00000100:00100000:3.0:1789760795.101593:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776316727314 time 776359567435 state 0x0 ref 1 00000100:00100000:3.0:1789760795.101608:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776359827851 time 776359567435 state 0x1 ref 1 00000100:00100000:3.0:1789760795.101646:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776359827851 time 776416948358 state 0x0 ref 1 00000100:00100000:3.0:1789760795.101658:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390848:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.101747:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24092672, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:3.0:1789760795.101757:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.101773:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.101778:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.101784:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.101799:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.135745:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00002000:00010000:2.0:1789760795.135873:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24092672 -> 24096768 00010000:00010000:2.0:1789760795.135887:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135900:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135909:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135923:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.135928:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135937:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135957:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.135965:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9d11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.136057:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240feb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390848:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34397us (34676us total) trans 4294973202 rc 0/0 00000100:00100000:2.0:1789760795.136068:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5912 00000100:00100000:2.0:1789760795.136078:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776359827851 time 776416948358 state 0x2 ref 0 00000100:00100000:0.0:1789760795.141445:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.141624:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507390976 00000100:00100000:3.0:1789760795.141634:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507390976 00000100:00100000:3.0:1789760795.141823:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776359827851 time 776416948358 state 0x0 ref 1 00000100:00100000:3.0:1789760795.141841:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776417208774 time 776416948358 state 0x1 ref 1 00000100:00100000:3.0:1789760795.141906:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776417208774 time 776457209628 state 0x0 ref 1 00000100:00100000:3.0:1789760795.141929:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390976:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.141990:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24096768, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:3.0:1789760795.141999:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.142015:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.142019:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.142025:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.142037:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.181771:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00002000:00010000:2.0:1789760795.181891:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:2.0:1789760795.181907:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.181932:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.181945:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.181954:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.181958:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.181968:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.182151:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.182161:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.182260:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c227868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507390976:12345-192.168.203.41@tcp:4:dd.0 Request processed in 40329us (40820us total) trans 4294973203 rc 0/0 00000100:00100000:2.0:1789760795.182270:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5913 00000100:00100000:2.0:1789760795.182281:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776417208774 time 776457209628 state 0x2 ref 0 00000100:00100000:3.0:1789760795.186828:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.187859:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391104 00000100:00100000:2.0:1789760795.187870:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391104 00000100:00100000:2.0:1789760795.188117:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776417208774 time 776457209628 state 0x0 ref 1 00000100:00100000:2.0:1789760795.188132:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776457470044 time 776457209628 state 0x1 ref 1 00000100:00100000:2.0:1789760795.188163:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776457470044 time 776503466197 state 0x0 ref 1 00000100:00100000:2.0:1789760795.188177:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391104:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.188242:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24100864, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:2.0:1789760795.188251:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.188265:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.188269:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.188274:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.188295:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.214272:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00002000:00010000:2.0:1789760795.214357:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:2.0:1789760795.214374:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214552:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214560:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214565:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.214570:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214576:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214594:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.214601:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.214803:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391104:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26623us (27978us total) trans 4294973204 rc 0/0 00000100:00100000:2.0:1789760795.214818:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5914 00000100:00100000:2.0:1789760795.214827:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776457470044 time 776503466197 state 0x2 ref 0 00000100:00100000:1.0:1789760795.218380:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.218598:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391232 00000100:00100000:2.0:1789760795.218609:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391232 00000100:00100000:2.0:1789760795.218661:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776457470044 time 776503466197 state 0x0 ref 1 00000100:00100000:2.0:1789760795.218677:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776503726613 time 776503466197 state 0x1 ref 1 00000100:00100000:2.0:1789760795.218737:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776503726613 time 776534039596 state 0x0 ref 1 00000100:00100000:2.0:1789760795.218750:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391232:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.218797:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24104960, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:2.0:1789760795.218807:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218822:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.218827:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.218834:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218849:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.244209:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:1.0:1789760795.244288:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24104960 -> 24109056 00010000:00010000:1.0:1789760795.244303:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244315:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244324:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244331:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760795.244335:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244343:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244362:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.244369:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf1ca9c1/0x9638aab0164b9d26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.244562:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391232:12345-192.168.203.41@tcp:4:dd.0 Request processed in 25808us (26190us total) trans 4294973205 rc 0/0 00000100:00100000:1.0:1789760795.244573:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5915 00000100:00100000:1.0:1789760795.244586:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776503726613 time 776534039596 state 0x2 ref 0 00000100:00100000:0.0:1789760795.250084:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.250228:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391360 00000100:00100000:2.0:1789760795.250236:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391360 00000100:00100000:2.0:1789760795.250281:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776503726613 time 776534039596 state 0x0 ref 1 00000100:00100000:2.0:1789760795.250293:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776534300012 time 776534039596 state 0x1 ref 1 00000100:00100000:2.0:1789760795.250341:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776534300012 time 776565645625 state 0x0 ref 1 00000100:00100000:2.0:1789760795.250350:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391360:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.250408:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24109056, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:2.0:1789760795.250416:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.250439:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.250450:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.250457:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.250470:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.278429:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00002000:00010000:3.0:1789760795.278625:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:3.0:1789760795.278639:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278652:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278661:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278669:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.278673:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278682:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278729:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.278737:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9d2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.278823:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391360:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28468us (28741us total) trans 4294973206 rc 0/0 00000100:00100000:3.0:1789760795.278834:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5916 00000100:00100000:3.0:1789760795.278843:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776534300012 time 776565645625 state 0x2 ref 0 00000100:00100000:3.0:1789760795.285970:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.286124:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391488 00000100:00100000:2.0:1789760795.286135:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391488 00000100:00100000:2.0:1789760795.286191:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776534300012 time 776565645625 state 0x0 ref 1 00000100:00100000:2.0:1789760795.286208:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776565906041 time 776565645625 state 0x1 ref 1 00000100:00100000:2.0:1789760795.286287:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776565906041 time 776601589510 state 0x0 ref 1 00000100:00100000:2.0:1789760795.286301:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507391488:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.286349:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24113152, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:2.0:1789760795.286358:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.286373:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.286377:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.286383:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.286407:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.329222:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:3.0:1789760795.329316:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24113152 -> 24117248 00010000:00010000:3.0:1789760795.329331:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329345:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329355:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329362:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.329367:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329377:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329406:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.329415:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9d34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.329509:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507391488:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43207us (43544us total) trans 4294973207 rc 0/0 00000100:00100000:3.0:1789760795.329523:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5917 00000100:00100000:3.0:1789760795.329533:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776565906041 time 776601589510 state 0x2 ref 0 00000100:00100000:0.0:1789760795.333524:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760795.336053:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391616 00000100:00100000:1.0:1789760795.336069:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391616 00000100:00100000:2.0:1789760795.336210:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776565906041 time 776601589510 state 0x0 ref 1 00000100:00100000:2.0:1789760795.336226:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776601849926 time 776601589510 state 0x1 ref 1 00000100:00100000:2.0:1789760795.336302:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776601849926 time 776651603146 state 0x0 ref 1 00000100:00100000:2.0:1789760795.336316:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391616:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.336366:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24117248, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xb807 00010000:00010000:2.0:1789760795.336375:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.336399:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.336404:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.336412:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.336426:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.370296:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00002000:00010000:2.0:1789760795.370417:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:2.0:1789760795.370487:0:14618:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789760795.370503:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370516:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370525:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370533:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.370537:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370546:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370568:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.370576:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.370751:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c22d5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391616:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34432us (37233us total) trans 4294973208 rc 0/0 00000100:00100000:2.0:1789760795.370762:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5918 00000100:00100000:2.0:1789760795.370772:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776601849926 time 776651603146 state 0x2 ref 0 00000100:00100000:0.0:1789760795.375759:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.375930:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391744 00000100:00100000:3.0:1789760795.375949:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391744 00000100:00100000:3.0:1789760795.376001:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776601849926 time 776651603146 state 0x0 ref 1 00000100:00100000:3.0:1789760795.376018:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776651863562 time 776651603146 state 0x1 ref 1 00000100:00100000:3.0:1789760795.376050:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776651863562 time 776691353533 state 0x0 ref 1 00000100:00100000:3.0:1789760795.376063:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507391744:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.376115:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24121344, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760795.376124:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.376140:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.376145:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.376152:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.376166:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.400303:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:3.0:1789760795.400321:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400332:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400338:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400344:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.400348:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400354:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400371:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.400375:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.400492:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507391744:12345-192.168.203.41@tcp:4:dd.0 Request processed in 24427us (24739us total) trans 4294973209 rc 0/0 00000100:00100000:3.0:1789760795.400504:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5919 00000100:00100000:3.0:1789760795.400513:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776651863562 time 776691353533 state 0x2 ref 0 00000001:00080000:3.0:1789760795.400556:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00000100:00100000:3.0:1789760795.413889:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.414166:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507391872 00000100:00100000:2.0:1789760795.414176:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507391872 00000100:00100000:2.0:1789760795.414235:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776651863562 time 776691353533 state 0x0 ref 1 00000100:00100000:2.0:1789760795.414250:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776691613949 time 776691353533 state 0x1 ref 1 00000100:00100000:2.0:1789760795.414353:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776691613949 time 776729594920 state 0x0 ref 1 00000100:00100000:2.0:1789760795.414370:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507391872:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.414439:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24125440, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:2.0:1789760795.414449:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.414465:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.414470:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.414477:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.414492:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.450500:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00002000:00010000:0.0:1789760795.450594:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:0.0:1789760795.450608:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450625:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450633:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450640:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760795.450644:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450653:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450671:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.450678:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9d49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.450802:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c104736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507391872:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36432us (36920us total) trans 4294973210 rc 0/0 00000100:00100000:0.0:1789760795.450812:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5920 00000100:00100000:0.0:1789760795.450822:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776691613949 time 776729594920 state 0x2 ref 0 00000100:00100000:0.0:1789760795.454669:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.458589:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392000 00000100:00100000:2.0:1789760795.458605:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392000 00000100:00100000:2.0:1789760795.458672:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776691613949 time 776729594920 state 0x0 ref 1 00000100:00100000:2.0:1789760795.458689:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776729855336 time 776729594920 state 0x1 ref 1 00000100:00100000:2.0:1789760795.458882:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776729855336 time 776774184154 state 0x0 ref 1 00000100:00100000:2.0:1789760795.458964:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392000:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.459014:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24129536, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:2.0:1789760795.459023:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.459039:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.459043:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.459049:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.459062:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760795.490565:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:2.0:1789760795.490584:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490597:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490605:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490613:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.490617:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490731:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490757:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.490766:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.490865:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507392000:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31966us (36201us total) trans 4294973211 rc 0/0 00000100:00100000:2.0:1789760795.490876:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5921 00000100:00100000:2.0:1789760795.490885:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776729855336 time 776774184154 state 0x2 ref 0 00000001:00080000:1.0:1789760795.490991:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00000100:00100000:1.0:1789760795.498908:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.499026:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392128 00000100:00100000:3.0:1789760795.499036:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392128 00000100:00100000:3.0:1789760795.499091:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776729855336 time 776774184154 state 0x0 ref 1 00000100:00100000:3.0:1789760795.499106:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776774444570 time 776774184154 state 0x1 ref 1 00000100:00100000:3.0:1789760795.499137:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776774444570 time 776814440753 state 0x0 ref 1 00000100:00100000:3.0:1789760795.499150:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392128:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.499194:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24133632, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760795.499203:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.499218:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.499222:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.499232:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.499246:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.537826:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00002000:00010000:3.0:1789760795.537954:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:3.0:1789760795.537971:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.537991:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.538001:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.538010:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.538016:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.538026:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.538048:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.538057:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.543033:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392128:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43830us (44090us total) trans 4294973212 rc 0/0 00000100:00100000:3.0:1789760795.543049:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5922 00000100:00100000:3.0:1789760795.543058:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776774444570 time 776814440753 state 0x2 ref 0 00010000:00010000:2.0:1789760795.548432:0:6429:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760795.548528:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.548598:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392256 00000100:00100000:3.0:1789760795.548607:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392256 00000100:00100000:3.0:1789760795.548671:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776774444570 time 776814440753 state 0x0 ref 1 00000100:00100000:3.0:1789760795.548686:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776814701169 time 776814440753 state 0x1 ref 1 00000100:00100000:3.0:1789760795.548876:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776814701169 time 776864035971 state 0x0 ref 1 00000100:00100000:3.0:1789760795.548890:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507392256:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.548944:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24137728, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760795.548953:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.548968:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.548972:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.548978:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.548991:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.581880:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00002000:00010000:3.0:1789760795.583542:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:3.0:1789760795.583561:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583580:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583636:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583644:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.583649:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583685:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583737:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.583745:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.583848:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507392256:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34956us (35321us total) trans 4294973213 rc 0/0 00000100:00100000:3.0:1789760795.583858:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5923 00000100:00100000:3.0:1789760795.583867:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776814701169 time 776864035971 state 0x2 ref 0 00000100:00100000:0.0:1789760795.588472:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.588615:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392384 00000100:00100000:2.0:1789760795.588629:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392384 00000100:00100000:2.0:1789760795.588729:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776814701169 time 776864035971 state 0x0 ref 1 00000100:00100000:2.0:1789760795.588753:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776864296387 time 776864035971 state 0x1 ref 1 00000100:00100000:2.0:1789760795.588771:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776864296387 time 776904075931 state 0x0 ref 1 00000100:00100000:2.0:1789760795.588783:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392384:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.588831:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24141824, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:2.0:1789760795.588841:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.588863:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760795.588867:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.588872:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.588885:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.624538:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:2.0:1789760795.624647:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:2.0:1789760795.624662:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624675:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624684:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624691:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760795.624714:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624724:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624744:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.624753:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9d65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.624882:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392384:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36096us (36417us total) trans 4294973214 rc 0/0 00000100:00100000:2.0:1789760795.624894:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5924 00000100:00100000:2.0:1789760795.624904:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776864296387 time 776904075931 state 0x2 ref 0 00000100:00100000:1.0:1789760795.636118:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.636205:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392512 00000100:00100000:3.0:1789760795.636220:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392512 00000100:00100000:3.0:1789760795.636274:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776864296387 time 776904075931 state 0x0 ref 1 00000100:00100000:3.0:1789760795.636293:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776904336347 time 776904075931 state 0x1 ref 1 00000100:00100000:3.0:1789760795.636325:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776904336347 time 776951627239 state 0x0 ref 1 00000100:00100000:3.0:1789760795.636338:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507392512:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.636396:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24145920, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760795.636405:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.636422:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760795.636428:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.636435:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.636448:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.698439:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:3.0:1789760795.698606:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24145920 -> 24150016 00010000:00010000:3.0:1789760795.698622:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698637:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698646:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698659:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760795.698664:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698672:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698729:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.698738:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.698831:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23b459500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507392512:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62490us (62719us total) trans 4294973215 rc 0/0 00000100:00100000:3.0:1789760795.698842:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5925 00000100:00100000:3.0:1789760795.698851:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776904336347 time 776951627239 state 0x2 ref 0 00000100:00100000:0.0:1789760795.707306:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.707369:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392640 00000100:00100000:3.0:1789760795.707380:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392640 00000100:00100000:3.0:1789760795.707456:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776904336347 time 776951627239 state 0x0 ref 1 00000100:00100000:3.0:1789760795.707471:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776951887655 time 776951627239 state 0x1 ref 1 00000100:00100000:3.0:1789760795.707498:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776951887655 time 777022801922 state 0x0 ref 1 00000100:00100000:3.0:1789760795.707514:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507392640:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760795.707563:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24150016, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760795.707572:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.707589:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760795.707594:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.707601:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.707626:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.762925:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24150016 -> 24154112 00010000:00010000:0.0:1789760795.762948:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.762962:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.762972:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.762980:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760795.762986:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.762996:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.763016:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.763024:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003136fca6/0x9638aab0164b9d73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.763122:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507392640:12345-192.168.203.41@tcp:4:dd.0 Request processed in 55606us (55822us total) trans 4294973216 rc 0/0 00000100:00100000:0.0:1789760795.763133:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5926 00000100:00100000:0.0:1789760795.763142:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776951887655 time 777022801922 state 0x2 ref 0 00000001:00080000:1.0:1789760795.763656:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00000100:00100000:1.0:1789760795.768343:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760795.768580:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392768 00000100:00100000:2.0:1789760795.768591:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392768 00000100:00100000:2.0:1789760795.768657:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 776951887655 time 777022801922 state 0x0 ref 1 00000100:00100000:2.0:1789760795.768672:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777023062338 time 777022801922 state 0x1 ref 1 00000100:00100000:2.0:1789760795.768886:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777023062338 time 777084186881 state 0x0 ref 1 00000100:00100000:2.0:1789760795.768901:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392768:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.768955:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24154112, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:2.0:1789760795.768964:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.768980:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760795.768985:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.768991:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.769009:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.834317:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00002000:00010000:0.0:1789760795.834478:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:0.0:1789760795.834494:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834507:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834516:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834524:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760795.834528:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834537:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834557:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.834565:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9d7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.834662:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507392768:12345-192.168.203.41@tcp:4:dd.0 Request processed in 65762us (66326us total) trans 4294973217 rc 0/0 00000100:00100000:0.0:1789760795.834670:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5927 00000100:00100000:0.0:1789760795.834679:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777023062338 time 777084186881 state 0x2 ref 0 00000100:00100000:2.0:1789760795.842665:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760795.844231:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507392896 00000100:00100000:3.0:1789760795.844244:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507392896 00000100:00100000:2.0:1789760795.844663:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777023062338 time 777084186881 state 0x0 ref 1 00000100:00100000:2.0:1789760795.844679:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777084447297 time 777084186881 state 0x1 ref 1 00000100:00100000:2.0:1789760795.844777:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777084447297 time 777160079198 state 0x0 ref 1 00000100:00100000:2.0:1789760795.844792:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507392896:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760795.844839:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24158208, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:2.0:1789760795.844847:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.844862:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760795.844867:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.844872:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.844887:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.868880:0:3463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1083eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538299392:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760795.868960:0:3463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.869840:0:8088:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299392 00000100:00100000:2.0:1789760795.869849:0:8088:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299392 00000100:00100000:2.0:1789760795.869898:0:8088:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1789760795.869908:0:8088:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538299392:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760795.869931:0:8088:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3834642432 avail 3805282304 00000100:00100000:2.0:1789760795.869985:0:8088:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3463:x1876699538299392:12345-0@lo:13:osp-pre-0-1.0 Request processed in 78us (1024us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760795.869993:0:8088:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1789760795.870059:0:3464:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538299520:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760795.870090:0:3464:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.870164:0:3463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1083eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3463:1876699538299392:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760795.870282:0:8087:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299520 00000100:00100000:0.0:1789760795.870287:0:8087:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299520 00000100:00100000:0.0:1789760795.870315:0:8087:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789760795.870322:0:8087:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538299520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760795.870334:0:8087:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3834642432 avail 3805282304 00000100:00100000:0.0:1789760795.870396:0:8087:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3464:x1876699538299520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.870402:0:8087:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789760795.870658:0:3464:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c234d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3464:1876699538299520:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789760795.878206:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24158208 -> 24162304 00010000:00010000:0.0:1789760795.878227:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878238:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878246:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878254:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760795.878259:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.878261:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00010000:00010000:0.0:1789760795.878273:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878296:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878304:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9d81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.878407:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507392896:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33610us (35744us total) trans 4294973218 rc 0/0 00000100:00100000:0.0:1789760795.878417:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5928 00000100:00100000:0.0:1789760795.878497:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777084447297 time 777160079198 state 0x2 ref 0 00000100:00100000:3.0:1789760795.882373:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760795.882725:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393024 00000100:00100000:0.0:1789760795.882736:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393024 00000100:00100000:0.0:1789760795.882792:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777084447297 time 777160079198 state 0x0 ref 1 00000100:00100000:0.0:1789760795.882807:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777160339614 time 777160079198 state 0x1 ref 1 00000100:00100000:1.0:1789760795.882880:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777160339614 time 777198180534 state 0x0 ref 1 00000100:00100000:1.0:1789760795.882896:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393024:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760795.882941:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24162304, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:1.0:1789760795.882950:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.882964:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760795.882968:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.882974:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.882987:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.968861:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:1.0:1789760795.968880:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968899:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968908:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968915:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760795.968921:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968930:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968949:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.968958:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.969053:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2275cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507393024:12345-192.168.203.41@tcp:4:dd.0 Request processed in 86156us (86685us total) trans 4294973219 rc 0/0 00000100:00100000:1.0:1789760795.969063:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5929 00000100:00100000:1.0:1789760795.969073:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777160339614 time 777198180534 state 0x2 ref 0 00000001:00080000:0.0:1789760795.969216:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00000100:00100000:1.0:1789760795.977887:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760795.980624:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393152 00000100:00100000:0.0:1789760795.980640:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393152 00000100:00100000:1.0:1789760795.983606:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777160339614 time 777198180534 state 0x0 ref 1 00000100:00100000:1.0:1789760795.983625:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777198440950 time 777198180534 state 0x1 ref 1 00000100:00100000:1.0:1789760795.983656:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777198440950 time 777298959897 state 0x0 ref 1 00000100:00100000:1.0:1789760795.983671:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507393152:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760795.983750:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24166400, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:1.0:1789760795.983759:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.983774:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760795.983779:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.983786:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.983801:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.042366:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00002000:00010000:2.0:1789760796.042555:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:2.0:1789760796.042575:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042590:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042599:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042607:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.042611:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042621:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042652:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.042661:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff2f9133/0x9638aab0164b9d8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.042839:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507393152:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59157us (64948us total) trans 4294973220 rc 0/0 00000100:00100000:2.0:1789760796.042852:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5930 00000100:00100000:2.0:1789760796.042865:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777198440950 time 777298959897 state 0x2 ref 0 00000100:00100000:2.0:1789760796.046059:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.052728:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393280 00000100:00100000:0.0:1789760796.052747:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393280 00000100:00100000:3.0:1789760796.056395:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777198440950 time 777298959897 state 0x0 ref 1 00000100:00100000:3.0:1789760796.056419:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777299220313 time 777298959897 state 0x1 ref 1 00000100:00100000:3.0:1789760796.056459:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777299220313 time 777371761843 state 0x0 ref 1 00000100:00100000:3.0:1789760796.056477:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393280:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.056543:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24170496, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc007 00010000:00010000:3.0:1789760796.056555:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.056576:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760796.056581:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.056589:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.056606:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.153652:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24170496 -> 24174592 00002000:00010000:0.0:1789760796.153663:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760796.153666:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00010000:00010000:0.0:1789760796.153682:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153727:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153737:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153746:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760796.153751:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153761:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153785:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.153793:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8ae8474/0x9638aab0164b9d96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.154020:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507393280:12345-192.168.203.41@tcp:4:dd.0 Request processed in 97541us (107968us total) trans 4294973221 rc 0/0 00000100:00100000:0.0:1789760796.154032:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5931 00000100:00100000:0.0:1789760796.154044:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777299220313 time 777371761843 state 0x2 ref 0 00000001:00080000:1.0:1789760796.155199:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00000100:00100000:1.0:1789760796.157202:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760796.157997:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393408 00000100:00100000:2.0:1789760796.158015:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393408 00000100:00100000:3.0:1789760796.158181:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777299220313 time 777371761843 state 0x0 ref 1 00000100:00100000:3.0:1789760796.158201:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777372022259 time 777371761843 state 0x1 ref 1 00000100:00100000:3.0:1789760796.158273:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777372022259 time 777473575846 state 0x0 ref 1 00000100:00100000:3.0:1789760796.158289:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507393408:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.158355:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24174592, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:3.0:1789760796.158365:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.158394:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760796.158399:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.158407:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.158435:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760796.210601:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:2.0:1789760796.210625:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210642:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210653:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210662:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.210667:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210680:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210737:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.210747:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029227a35/0x9638aab0164b9d9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.210959:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00000100:00100000:2.0:1789760796.211311:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507393408:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52569us (53665us total) trans 4294973222 rc 0/0 00000100:00100000:2.0:1789760796.211324:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5932 00000100:00100000:2.0:1789760796.211336:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777372022259 time 777473575846 state 0x2 ref 0 00000100:00100000:3.0:1789760796.213955:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760796.214144:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393536 00000100:00100000:2.0:1789760796.214156:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393536 00000100:00100000:2.0:1789760796.214226:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777372022259 time 777473575846 state 0x0 ref 1 00000100:00100000:2.0:1789760796.214245:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777473836262 time 777473575846 state 0x1 ref 1 00000100:00100000:2.0:1789760796.214318:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777473836262 time 777529620879 state 0x0 ref 1 00000100:00100000:2.0:1789760796.214333:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507393536:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760796.214410:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24178688, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:2.0:1789760796.214429:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.214451:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760796.214456:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.214464:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.214481:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.286308:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:3.0:1789760796.286533:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:3.0:1789760796.286553:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286569:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286580:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286589:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760796.286595:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286606:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286630:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.286639:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007369b58b/0x9638aab0164b9da4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.287319:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507393536:12345-192.168.203.41@tcp:4:dd.0 Request processed in 72976us (73367us total) trans 4294973223 rc 0/0 00000100:00100000:2.0:1789760796.287334:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5933 00000100:00100000:2.0:1789760796.287347:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777473836262 time 777529620879 state 0x2 ref 0 00000100:00100000:3.0:1789760796.290116:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.290299:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393664 00000100:00100000:0.0:1789760796.290314:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393664 00000100:00100000:0.0:1789760796.290407:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777473836262 time 777529620879 state 0x0 ref 1 00000100:00100000:0.0:1789760796.290435:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777529881295 time 777529620879 state 0x1 ref 1 00000100:00100000:0.0:1789760796.290473:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777529881295 time 777605776076 state 0x0 ref 1 00000100:00100000:0.0:1789760796.290488:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393664:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.290563:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24182784, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.290574:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.290595:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760796.290601:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.290609:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.290626:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.324726:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00002000:00010000:3.0:1789760796.324933:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:3.0:1789760796.324953:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.324968:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.324979:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.324988:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760796.324993:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.325004:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.325029:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.325038:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c986e0/0x9638aab0164b9dab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.325183:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393664:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34692us (35078us total) trans 4294973224 rc 0/0 00000100:00100000:3.0:1789760796.325194:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5934 00000100:00100000:3.0:1789760796.325204:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777529881295 time 777605776076 state 0x2 ref 0 00000100:00100000:2.0:1789760796.328964:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760796.329113:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393792 00000100:00100000:3.0:1789760796.329128:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393792 00000100:00100000:3.0:1789760796.329208:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777529881295 time 777605776076 state 0x0 ref 1 00000100:00100000:3.0:1789760796.329228:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777606036492 time 777605776076 state 0x1 ref 1 00000100:00100000:3.0:1789760796.329266:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777606036492 time 777644568011 state 0x0 ref 1 00000100:00100000:3.0:1789760796.329280:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507393792:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.329344:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24186880, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:3.0:1789760796.329355:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.329375:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760796.329390:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.329399:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.329423:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.375129:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00002000:00010000:3.0:1789760796.375239:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24186880 -> 24190976 00010000:00010000:3.0:1789760796.375260:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375274:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375285:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375294:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760796.375299:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375308:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375335:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.375342:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9db2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.375639:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507393792:12345-192.168.203.41@tcp:4:dd.0 Request processed in 46352us (46680us total) trans 4294973225 rc 0/0 00000100:00100000:3.0:1789760796.375652:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5935 00000100:00100000:3.0:1789760796.375665:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777606036492 time 777644568011 state 0x2 ref 0 00000100:00100000:2.0:1789760796.378584:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.378956:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507393920 00000100:00100000:0.0:1789760796.378970:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507393920 00000100:00100000:1.0:1789760796.381999:0:3462:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760796.382032:0:3462:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c1083eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538299648:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760796.382125:0:3462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760796.382217:0:6445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299648 00000100:00100000:1.0:1789760796.382226:0:6445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299648 00000100:00100000:1.0:1789760796.382292:0:6445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1789760796.382303:0:6445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538299648:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760796.382355:0:6445:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760796.382432:0:6445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3462:x1876699538299648:12345-0@lo:41:osp-pre-1-0.0 Request processed in 130us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760796.382441:0:6445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1789760796.382497:0:3462:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c1083eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3462:1876699538299648:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760796.383225:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777606036492 time 777644568011 state 0x0 ref 1 00000100:00100000:2.0:1789760796.383247:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777644828427 time 777644568011 state 0x1 ref 1 00000100:00100000:2.0:1789760796.383279:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777644828427 time 777698582314 state 0x0 ref 1 00000100:00100000:2.0:1789760796.383290:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393920:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760796.383345:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24190976, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:2.0:1789760796.383357:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.383472:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760796.383479:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.383487:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.383505:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.425472:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00002000:00010000:0.0:1789760796.425762:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:0.0:1789760796.425850:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.425867:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.425877:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.425887:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760796.426119:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.426133:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.426161:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.426169:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e38b31f4/0x9638aab0164b9db9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.426352:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507393920:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43056us (47775us total) trans 4294973226 rc 0/0 00000100:00100000:0.0:1789760796.426436:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5936 00000100:00100000:0.0:1789760796.426448:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777644828427 time 777698582314 state 0x2 ref 0 00000100:00100000:2.0:1789760796.428939:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760796.429097:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394048 00000100:00100000:3.0:1789760796.429110:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394048 00000100:00100000:3.0:1789760796.429180:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777644828427 time 777698582314 state 0x0 ref 1 00000100:00100000:3.0:1789760796.429198:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777698842730 time 777698582314 state 0x1 ref 1 00000100:00100000:3.0:1789760796.429235:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777698842730 time 777744537788 state 0x0 ref 1 00000100:00100000:3.0:1789760796.429249:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507394048:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.429314:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24195072, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:3.0:1789760796.429324:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429343:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760796.429348:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.429355:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429371:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.481301:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00002000:00010000:0.0:1789760796.481527:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:0.0:1789760796.481548:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481565:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481576:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481584:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a2600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760796.481589:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481600:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481624:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481631:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076022872/0x9638aab0164b9dc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.481853:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507394048:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52600us (52969us total) trans 4294973227 rc 0/0 00000100:00100000:0.0:1789760796.481869:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5937 00000100:00100000:0.0:1789760796.481881:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777698842730 time 777744537788 state 0x2 ref 0 00000100:00100000:3.0:1789760796.484660:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760796.484912:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394176 00000100:00100000:2.0:1789760796.484926:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394176 00000100:00100000:2.0:1789760796.493939:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777698842730 time 777744537788 state 0x0 ref 1 00000100:00100000:2.0:1789760796.493969:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777744798204 time 777744537788 state 0x1 ref 1 00000100:00100000:2.0:1789760796.494012:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777744798204 time 777809315178 state 0x0 ref 1 00000100:00100000:2.0:1789760796.494039:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394176:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760796.494094:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24199168, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:2.0:1789760796.494103:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.494118:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760796.494121:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.494128:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.494141:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.553023:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00002000:00010000:2.0:1789760796.553199:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:2.0:1789760796.553219:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553236:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553247:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553254:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.553260:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553271:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553296:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.553303:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9dc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.553519:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394176:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59488us (68865us total) trans 4294973228 rc 0/0 00000100:00100000:2.0:1789760796.553532:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5938 00000100:00100000:2.0:1789760796.553545:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777744798204 time 777809315178 state 0x2 ref 0 00000100:00100000:2.0:1789760796.556592:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.556788:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394304 00000100:00100000:0.0:1789760796.556800:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394304 00000100:00100000:0.0:1789760796.556925:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777744798204 time 777809315178 state 0x0 ref 1 00000100:00100000:0.0:1789760796.556943:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777809575594 time 777809315178 state 0x1 ref 1 00000100:00100000:0.0:1789760796.557007:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777809575594 time 777872309775 state 0x0 ref 1 00000100:00100000:0.0:1789760796.557020:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394304:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.557083:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24203264, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.557093:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.557112:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760796.557117:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.557125:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.557142:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.572449:0:9034:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760796.591099:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24203264 -> 24207360 00010000:00010000:2.0:1789760796.591125:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591141:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591152:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591160:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.591165:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591174:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591201:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.591216:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f413fe3d/0x9638aab0164b9dce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.591337:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507394304:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34311us (34753us total) trans 4294973229 rc 0/0 00000100:00100000:2.0:1789760796.591351:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5939 00000100:00100000:2.0:1789760796.591363:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777809575594 time 777872309775 state 0x2 ref 0 00000001:00080000:1.0:1789760796.593369:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00000100:00100000:2.0:1789760796.594357:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.594580:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394432 00000100:00100000:0.0:1789760796.594593:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394432 00000100:00100000:0.0:1789760796.594767:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777809575594 time 777872309775 state 0x0 ref 1 00000100:00100000:0.0:1789760796.594785:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777872570191 time 777872309775 state 0x1 ref 1 00000100:00100000:0.0:1789760796.594820:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777872570191 time 777910122782 state 0x0 ref 1 00000100:00100000:0.0:1789760796.594836:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23473b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394432:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.594906:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24207360, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.594917:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.594937:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760796.594942:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.594950:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.594968:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760796.638145:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:2.0:1789760796.638171:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638188:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638197:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638206:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.638212:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638222:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638251:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.638260:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003c356eb/0x9638aab0164b9dd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.638392:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23473b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507394432:12345-192.168.203.41@tcp:4:dd.0 Request processed in 43545us (44031us total) trans 4294973230 rc 0/0 00000100:00100000:2.0:1789760796.638404:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5940 00000100:00100000:2.0:1789760796.638485:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777872570191 time 777910122782 state 0x2 ref 0 00000001:00080000:3.0:1789760796.641308:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00000100:00100000:2.0:1789760796.641512:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760796.641608:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394560 00000100:00100000:3.0:1789760796.641620:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394560 00000100:00100000:3.0:1789760796.641689:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777872570191 time 777910122782 state 0x0 ref 1 00000100:00100000:3.0:1789760796.641783:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777910383198 time 777910122782 state 0x1 ref 1 00000100:00100000:3.0:1789760796.641817:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777910383198 time 777957119780 state 0x0 ref 1 00000100:00100000:3.0:1789760796.641985:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394560:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.642049:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24211456, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:3.0:1789760796.642060:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.642080:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760796.642085:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.642092:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.642110:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.676957:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:3.0:1789760796.677087:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:3.0:1789760796.677107:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677123:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677133:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677141:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760796.677146:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677156:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677180:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.677188:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9ddc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.677314:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394560:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35327us (35800us total) trans 4294973231 rc 0/0 00000100:00100000:3.0:1789760796.677325:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5941 00000100:00100000:3.0:1789760796.677338:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777910383198 time 777957119780 state 0x2 ref 0 00000100:00100000:0.0:1789760796.680954:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760796.681081:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394688 00000100:00100000:2.0:1789760796.681094:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394688 00000100:00100000:2.0:1789760796.681171:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777910383198 time 777957119780 state 0x0 ref 1 00000100:00100000:2.0:1789760796.681189:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777957380196 time 777957119780 state 0x1 ref 1 00000100:00100000:2.0:1789760796.681255:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777957380196 time 777996557129 state 0x0 ref 1 00000100:00100000:2.0:1789760796.681270:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394688:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760796.681335:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24215552, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:2.0:1789760796.681346:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.681371:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760796.681387:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.681395:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.681410:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.712396:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00002000:00010000:2.0:1789760796.712554:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:2.0:1789760796.712574:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712589:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712600:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712608:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5a00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760796.712614:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712625:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712661:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.712671:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aaeb146/0x9638aab0164b9de3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.713046:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394688:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31554us (31880us total) trans 4294973232 rc 0/0 00000100:00100000:2.0:1789760796.713060:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5942 00000100:00100000:2.0:1789760796.713074:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777957380196 time 777996557129 state 0x2 ref 0 00000100:00100000:0.0:1789760796.715898:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760796.715984:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394816 00000100:00100000:2.0:1789760796.715996:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394816 00000100:00100000:2.0:1789760796.716062:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777957380196 time 777996557129 state 0x0 ref 1 00000100:00100000:2.0:1789760796.716081:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777996817545 time 777996557129 state 0x1 ref 1 00000100:00100000:2.0:1789760796.716155:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777996817545 time 778031457708 state 0x0 ref 1 00000100:00100000:2.0:1789760796.716169:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507394816:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760796.716231:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24219648, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:2.0:1789760796.716242:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.716262:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760796.716268:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.716276:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.716294:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.768723:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:0.0:1789760796.768747:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:0.0:1789760796.768768:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768784:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768800:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768808:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760796.768814:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768824:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768849:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768859:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e6859af/0x9638aab0164b9dea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.769017:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507394816:12345-192.168.203.41@tcp:4:dd.0 Request processed in 52841us (53125us total) trans 4294973233 rc 0/0 00000100:00100000:0.0:1789760796.769030:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5943 00000100:00100000:0.0:1789760796.769043:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777996817545 time 778031457708 state 0x2 ref 0 00000100:00100000:2.0:1789760796.771835:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760796.771942:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507394944 00000100:00100000:3.0:1789760796.771955:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507394944 00000100:00100000:3.0:1789760796.772029:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 777996817545 time 778031457708 state 0x0 ref 1 00000100:00100000:3.0:1789760796.772048:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778031718124 time 778031457708 state 0x1 ref 1 00000100:00100000:3.0:1789760796.772080:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778031718124 time 778087383027 state 0x0 ref 1 00000100:00100000:3.0:1789760796.772094:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394944:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760796.772158:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24223744, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:3.0:1789760796.772169:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.772189:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760796.772208:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.772217:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.772234:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.801488:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00002000:00010000:0.0:1789760796.802023:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:0.0:1789760796.802038:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802056:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802065:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802073:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760796.802078:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802088:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802108:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802116:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c550992f/0x9638aab0164b9df1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.802214:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234738700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507394944:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30117us (30399us total) trans 4294973234 rc 0/0 00000100:00100000:0.0:1789760796.802226:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5944 00000100:00100000:0.0:1789760796.802234:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778031718124 time 778087383027 state 0x2 ref 0 00000100:00100000:2.0:1789760796.807084:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.807148:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395072 00000100:00100000:0.0:1789760796.807158:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395072 00000100:00100000:0.0:1789760796.807214:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778031718124 time 778087383027 state 0x0 ref 1 00000100:00100000:0.0:1789760796.807228:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778087643443 time 778087383027 state 0x1 ref 1 00000100:00100000:0.0:1789760796.807317:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778087643443 time 778122619496 state 0x0 ref 1 00000100:00100000:0.0:1789760796.807330:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395072:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.807389:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24227840, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.807398:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.807414:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760796.807419:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.807425:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.807439:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.863644:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:1.0:1789760796.863909:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:1.0:1789760796.863924:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863936:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863946:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863953:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760796.863957:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863966:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863985:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.863993:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9df8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.864072:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395072:12345-192.168.203.41@tcp:4:dd.0 Request processed in 56740us (56993us total) trans 4294973235 rc 0/0 00000100:00100000:1.0:1789760796.864082:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5945 00000100:00100000:1.0:1789760796.864093:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778087643443 time 778122619496 state 0x2 ref 0 00000100:00100000:2.0:1789760796.871999:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.872250:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395200 00000100:00100000:0.0:1789760796.872261:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395200 00000100:00100000:0.0:1789760796.872317:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778087643443 time 778122619496 state 0x0 ref 1 00000100:00100000:0.0:1789760796.872333:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778122879912 time 778122619496 state 0x1 ref 1 00000100:00100000:0.0:1789760796.872416:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778122879912 time 778187718683 state 0x0 ref 1 00000100:00100000:0.0:1789760796.872443:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395200:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.872490:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24231936, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.872499:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.872515:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760796.872520:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.872525:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.872540:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.894747:0:3461:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760796.894771:0:3461:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff88c21f9d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538299776:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760796.894841:0:3461:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760796.894916:0:6446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699538299776 00000100:00100000:3.0:1789760796.894923:0:6446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699538299776 00000100:00100000:3.0:1789760796.894963:0:6446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1789760796.894972:0:6446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538299776:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760796.895006:0:6446:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760796.895056:0:6446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3461:x1876699538299776:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760796.895063:0:6446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1789760796.895112:0:3461:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff88c21f9d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3461:1876699538299776:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789760796.933978:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00002000:00010000:1.0:1789760796.934155:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:1.0:1789760796.934171:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934186:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934195:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934202:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760796.934207:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934222:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934245:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934252:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9dff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.934349:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395200:12345-192.168.203.41@tcp:4:dd.0 Request processed in 61904us (62355us total) trans 4294973236 rc 0/0 00000100:00100000:1.0:1789760796.934359:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5946 00000100:00100000:1.0:1789760796.934370:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778122879912 time 778187718683 state 0x2 ref 0 00000100:00100000:3.0:1789760796.937295:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760796.937520:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395328 00000100:00100000:0.0:1789760796.937530:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395328 00000100:00100000:0.0:1789760796.937593:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778122879912 time 778187718683 state 0x0 ref 1 00000100:00100000:0.0:1789760796.937610:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778187979099 time 778187718683 state 0x1 ref 1 00000100:00100000:0.0:1789760796.937646:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778187979099 time 778252949877 state 0x0 ref 1 00000100:00100000:0.0:1789760796.937659:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395328:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760796.937801:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24236032, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:0.0:1789760796.937810:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.937903:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760796.937908:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.937914:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.937928:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.968790:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00002000:00010000:1.0:1789760796.968851:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:1.0:1789760796.968868:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968882:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968892:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968900:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760796.968905:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968915:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968935:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.968943:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9e06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.969043:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c21f9d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395328:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31381us (31753us total) trans 4294973237 rc 0/0 00000100:00100000:1.0:1789760796.969053:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5947 00000100:00100000:1.0:1789760796.969062:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778187979099 time 778252949877 state 0x2 ref 0 00000100:00100000:2.0:1789760796.972829:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760796.972937:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395456 00000100:00100000:1.0:1789760796.972946:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395456 00000100:00100000:1.0:1789760796.973001:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778187979099 time 778252949877 state 0x0 ref 1 00000100:00100000:1.0:1789760796.973017:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778253210293 time 778252949877 state 0x1 ref 1 00000100:00100000:1.0:1789760796.973074:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778253210293 time 778288376735 state 0x0 ref 1 00000100:00100000:1.0:1789760796.973087:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395456:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760796.973138:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24240128, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:1.0:1789760796.973147:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.973174:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760796.973179:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.973186:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.973200:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.002747:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00002000:00010000:1.0:1789760797.002855:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24240128 -> 24244224 00010000:00010000:1.0:1789760797.002872:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002885:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002895:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002902:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.002908:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002917:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002942:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.002950:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.003042:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395456:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29952us (30219us total) trans 4294973238 rc 0/0 00000100:00100000:1.0:1789760797.003052:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5948 00000100:00100000:1.0:1789760797.003062:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778253210293 time 778288376735 state 0x2 ref 0 00000100:00100000:2.0:1789760797.006414:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.006491:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395584 00000100:00100000:1.0:1789760797.006502:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395584 00000100:00100000:1.0:1789760797.006570:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778253210293 time 778288376735 state 0x0 ref 1 00000100:00100000:1.0:1789760797.006586:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778288637151 time 778288376735 state 0x1 ref 1 00000100:00100000:1.0:1789760797.006618:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778288637151 time 778321921462 state 0x0 ref 1 00000100:00100000:1.0:1789760797.006632:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395584:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.006680:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24244224, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc007 00010000:00010000:1.0:1789760797.006689:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.006738:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760797.006742:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.006749:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.006764:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760797.006981:0:6436:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760797.038729:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00002000:00010000:3.0:1789760797.038760:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24244224 -> 24248320 00002000:00010000:3.0:1789760797.038766:0:14616:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:3.0:1789760797.038769:0:14616:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:3.0:1789760797.038783:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038795:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038805:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038813:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760797.038913:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038924:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038946:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.038954:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022a33f97/0x9638aab0164b9e14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.039085:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507395584:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32449us (32675us total) trans 4294973239 rc 0/0 00000100:00100000:3.0:1789760797.039096:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5949 00000100:00100000:3.0:1789760797.039106:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778288637151 time 778321921462 state 0x2 ref 0 00000100:00100000:3.0:1789760797.042831:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.042903:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395712 00000100:00100000:1.0:1789760797.042913:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395712 00000100:00100000:1.0:1789760797.042966:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778288637151 time 778321921462 state 0x0 ref 1 00000100:00100000:1.0:1789760797.042981:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778322181878 time 778321921462 state 0x1 ref 1 00000100:00100000:1.0:1789760797.043012:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778322181878 time 778358315374 state 0x0 ref 1 00000100:00100000:1.0:1789760797.043025:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395712:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.043072:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24248320, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.043081:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.043097:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760797.043102:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.043108:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.043122:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.075122:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00002000:00010000:2.0:1789760797.075212:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24248320 -> 24252416 00010000:00010000:2.0:1789760797.075227:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075240:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075250:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075258:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dbc00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.075262:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075278:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075308:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.075317:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2b8e02/0x9638aab0164b9e1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.075428:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395712:12345-192.168.203.41@tcp:4:dd.0 Request processed in 32394us (32597us total) trans 4294973240 rc 0/0 00000100:00100000:2.0:1789760797.075438:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5950 00000100:00100000:2.0:1789760797.075447:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778322181878 time 778358315374 state 0x2 ref 0 00000100:00100000:2.0:1789760797.078253:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.078358:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395840 00000100:00100000:1.0:1789760797.078367:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395840 00000100:00100000:1.0:1789760797.078517:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778322181878 time 778358315374 state 0x0 ref 1 00000100:00100000:1.0:1789760797.078530:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778358575790 time 778358315374 state 0x1 ref 1 00000100:00100000:1.0:1789760797.078573:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778358575790 time 778393876904 state 0x0 ref 1 00000100:00100000:1.0:1789760797.078582:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507395840:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.078621:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24252416, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.078627:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.078639:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760797.078643:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.078648:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.078662:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.101002:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24252416 -> 24256512 00010000:00010000:2.0:1789760797.101024:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101037:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101046:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101058:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082dac00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.101064:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101072:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101091:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.101098:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec848e1d/0x9638aab0164b9e22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.101189:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507395840:12345-192.168.203.41@tcp:4:dd.0 Request processed in 22602us (22939us total) trans 4294973241 rc 0/0 00000100:00100000:2.0:1789760797.101199:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5951 00000100:00100000:2.0:1789760797.101209:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778358575790 time 778393876904 state 0x2 ref 0 00000001:00080000:3.0:1789760797.101511:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00000100:00100000:3.0:1789760797.114041:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760797.114264:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507395968 00000100:00100000:0.0:1789760797.114273:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507395968 00000100:00100000:0.0:1789760797.114321:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778358575790 time 778393876904 state 0x0 ref 1 00000100:00100000:0.0:1789760797.114332:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778394137320 time 778393876904 state 0x1 ref 1 00000100:00100000:0.0:1789760797.114414:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778394137320 time 778429716451 state 0x0 ref 1 00000100:00100000:0.0:1789760797.114428:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395968:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760797.114490:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24256512, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:0.0:1789760797.114499:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.114514:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760797.114518:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.114524:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.114549:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.166913:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:0.0:1789760797.167936:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:0.0:1789760797.167955:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.167970:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.167979:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.167987:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760797.167992:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.168001:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.168023:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.168030:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001740b623/0x9638aab0164b9e29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.168249:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507395968:12345-192.168.203.41@tcp:4:dd.0 Request processed in 53818us (54213us total) trans 4294973242 rc 0/0 00000100:00100000:0.0:1789760797.168259:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5952 00000100:00100000:0.0:1789760797.168269:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778394137320 time 778429716451 state 0x2 ref 0 00000100:00100000:1.0:1789760797.190669:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.190877:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396096 00000100:00100000:2.0:1789760797.190886:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396096 00000100:00100000:2.0:1789760797.190943:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778394137320 time 778429716451 state 0x0 ref 1 00000100:00100000:2.0:1789760797.190959:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778429976867 time 778429716451 state 0x1 ref 1 00000100:00100000:2.0:1789760797.190990:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778429976867 time 778506293725 state 0x0 ref 1 00000100:00100000:2.0:1789760797.191003:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507396096:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.191055:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24260608, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.191064:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191080:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.191085:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.191091:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191107:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.234353:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00002000:00010000:2.0:1789760797.235337:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:2.0:1789760797.235354:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235369:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235394:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235402:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.235540:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235551:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235572:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235580:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.235748:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507396096:12345-192.168.203.41@tcp:4:dd.0 Request processed in 44742us (45083us total) trans 4294973243 rc 0/0 00000100:00100000:2.0:1789760797.235758:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5953 00000100:00100000:2.0:1789760797.235768:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778429976867 time 778506293725 state 0x2 ref 0 00000100:00100000:0.0:1789760797.239256:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.239340:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396224 00000100:00100000:1.0:1789760797.239349:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396224 00000100:00100000:1.0:1789760797.239432:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778429976867 time 778506293725 state 0x0 ref 1 00000100:00100000:1.0:1789760797.239449:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778506554141 time 778506293725 state 0x1 ref 1 00000100:00100000:1.0:1789760797.239479:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778506554141 time 778554782570 state 0x0 ref 1 00000100:00100000:1.0:1789760797.239492:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c2414d6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507396224:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.239538:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24264704, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.239547:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.239561:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760797.239566:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.239573:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.239588:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.290636:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:1.0:1789760797.291016:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:1.0:1789760797.291031:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291044:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291053:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291066:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.291071:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291080:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291097:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.291104:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000114136be/0x9638aab0164b9e37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.291251:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c2414d6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507396224:12345-192.168.203.41@tcp:4:dd.0 Request processed in 51756us (51999us total) trans 4294973244 rc 0/0 00000100:00100000:1.0:1789760797.291261:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5954 00000100:00100000:1.0:1789760797.291270:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778506554141 time 778554782570 state 0x2 ref 0 00000100:00100000:0.0:1789760797.297263:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.297357:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396352 00000100:00100000:2.0:1789760797.297366:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396352 00000100:00100000:2.0:1789760797.297440:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778506554141 time 778554782570 state 0x0 ref 1 00000100:00100000:2.0:1789760797.297455:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778555042986 time 778554782570 state 0x1 ref 1 00000100:00100000:2.0:1789760797.297487:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778555042986 time 778612790320 state 0x0 ref 1 00000100:00100000:2.0:1789760797.297500:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507396352:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.297547:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24268800, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.297556:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.297573:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.297576:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.297582:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.297594:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.336233:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00002000:00010000:0.0:1789760797.336251:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:0.0:1789760797.336267:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336285:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336295:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336302:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760797.336307:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336315:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336340:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.336349:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b809ad61/0x9638aab0164b9e3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.336457:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108041500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507396352:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38954us (39197us total) trans 4294973245 rc 0/0 00000100:00100000:0.0:1789760797.336467:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5955 00000100:00100000:0.0:1789760797.336477:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778555042986 time 778612790320 state 0x2 ref 0 00000100:00100000:3.0:1789760797.342262:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.342517:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396480 00000100:00100000:2.0:1789760797.342528:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396480 00000100:00100000:2.0:1789760797.342586:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778555042986 time 778612790320 state 0x0 ref 1 00000100:00100000:2.0:1789760797.342858:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778613050736 time 778612790320 state 0x1 ref 1 00000100:00100000:2.0:1789760797.342880:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778613050736 time 778658184834 state 0x0 ref 1 00000100:00100000:2.0:1789760797.342893:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396480:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.342947:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24272896, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.342956:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342971:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.342976:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.342983:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.343000:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.387368:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00002000:00010000:1.0:1789760797.387596:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:1.0:1789760797.387617:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387631:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387647:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387654:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.387659:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387667:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387687:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387767:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bb74c6/0x9638aab0164b9e45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.387974:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234676d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396480:12345-192.168.203.41@tcp:4:dd.0 Request processed in 45077us (45714us total) trans 4294973246 rc 0/0 00000100:00100000:1.0:1789760797.387984:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5956 00000100:00100000:1.0:1789760797.387994:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778613050736 time 778658184834 state 0x2 ref 0 00000100:00100000:2.0:1789760797.392986:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760797.393156:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396608 00000100:00100000:0.0:1789760797.393166:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396608 00000100:00100000:0.0:1789760797.393224:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778613050736 time 778658184834 state 0x0 ref 1 00000100:00100000:0.0:1789760797.393240:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778658445250 time 778658184834 state 0x1 ref 1 00000100:00100000:0.0:1789760797.393303:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778658445250 time 778708605740 state 0x0 ref 1 00000100:00100000:0.0:1789760797.393316:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507396608:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760797.393367:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24276992, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:0.0:1789760797.393385:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.393402:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760797.393416:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.393423:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.393438:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760797.427293:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:0.0:1789760797.427312:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427326:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427336:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427344:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760797.427349:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427358:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427388:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.427400:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa4cbb32/0x9638aab0164b9e4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.427566:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507396608:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34181us (34520us total) trans 4294973247 rc 0/0 00000100:00100000:0.0:1789760797.427578:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5957 00000100:00100000:0.0:1789760797.427588:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778658445250 time 778708605740 state 0x2 ref 0 00000001:00080000:2.0:1789760797.427730:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00000100:00100000:0.0:1789760797.432429:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.432608:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396736 00000100:00100000:2.0:1789760797.432617:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396736 00000100:00100000:2.0:1789760797.432673:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778658445250 time 778708605740 state 0x0 ref 1 00000100:00100000:2.0:1789760797.432686:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778708866156 time 778708605740 state 0x1 ref 1 00000100:00100000:2.0:1789760797.432735:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778708866156 time 778748038643 state 0x0 ref 1 00000100:00100000:2.0:1789760797.432746:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396736:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.432796:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24281088, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.432805:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.432820:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.432825:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.432830:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.432853:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.472003:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00002000:00010000:3.0:1789760797.472184:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:3.0:1789760797.472201:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472216:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472226:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472234:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa4c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760797.472239:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472249:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472270:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.472279:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed2d998e/0x9638aab0164b9e53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.472384:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c23462bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396736:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39625us (39951us total) trans 4294973248 rc 0/0 00000100:00100000:3.0:1789760797.472395:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5958 00000100:00100000:3.0:1789760797.472475:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778708866156 time 778748038643 state 0x2 ref 0 00000100:00100000:1.0:1789760797.475430:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.475603:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396864 00000100:00100000:2.0:1789760797.475613:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396864 00000100:00100000:2.0:1789760797.475669:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778708866156 time 778748038643 state 0x0 ref 1 00000100:00100000:2.0:1789760797.475684:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778748299059 time 778748038643 state 0x1 ref 1 00000100:00100000:2.0:1789760797.475739:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778748299059 time 778791041650 state 0x0 ref 1 00000100:00100000:2.0:1789760797.475757:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507396864:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.475807:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24285184, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.475816:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.475830:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.475834:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.475841:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.475854:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.507177:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00002000:00010000:2.0:1789760797.507281:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24285184 -> 24289280 00010000:00010000:2.0:1789760797.507297:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507316:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507326:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507333:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.507339:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507348:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507380:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.507389:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.507491:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507396864:12345-192.168.203.41@tcp:4:dd.0 Request processed in 31732us (32068us total) trans 4294973249 rc 0/0 00000100:00100000:2.0:1789760797.507503:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5959 00000100:00100000:2.0:1789760797.507513:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778748299059 time 778791041650 state 0x2 ref 0 00000100:00100000:3.0:1789760797.511026:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.511222:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507396992 00000100:00100000:1.0:1789760797.511232:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507396992 00000100:00100000:1.0:1789760797.511281:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778748299059 time 778791041650 state 0x0 ref 1 00000100:00100000:1.0:1789760797.511294:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778791302066 time 778791041650 state 0x1 ref 1 00000100:00100000:1.0:1789760797.511318:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778791302066 time 778826621915 state 0x0 ref 1 00000100:00100000:1.0:1789760797.511329:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396992:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.511389:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24289280, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.511398:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.511417:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760797.511422:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.511428:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.511441:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.569346:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00002000:00010000:1.0:1789760797.569487:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:1.0:1789760797.569503:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569515:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569530:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569537:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.569542:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569551:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569572:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.569579:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.569811:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234677b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507396992:12345-192.168.203.41@tcp:4:dd.0 Request processed in 58477us (58790us total) trans 4294973250 rc 0/0 00000100:00100000:1.0:1789760797.569890:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5960 00000100:00100000:1.0:1789760797.569900:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778791302066 time 778826621915 state 0x2 ref 0 00000100:00100000:3.0:1789760797.572921:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.573038:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397120 00000100:00100000:2.0:1789760797.573048:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397120 00000100:00100000:2.0:1789760797.573102:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778791302066 time 778826621915 state 0x0 ref 1 00000100:00100000:2.0:1789760797.573117:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778826882331 time 778826621915 state 0x1 ref 1 00000100:00100000:2.0:1789760797.573177:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778826882331 time 778888479437 state 0x0 ref 1 00000100:00100000:2.0:1789760797.573191:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507397120:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.573239:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24293376, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.573248:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.573264:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.573268:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.573275:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.573289:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.595900:0:9035:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760797.611240:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00002000:00010000:1.0:1789760797.611725:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24293376 -> 24297472 00010000:00010000:1.0:1789760797.611753:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611772:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611781:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611788:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.611793:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611803:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611823:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.611830:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001045e799/0x9638aab0164b9e68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.611924:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108017800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507397120:12345-192.168.203.41@tcp:4:dd.0 Request processed in 38731us (39008us total) trans 4294973251 rc 0/0 00000100:00100000:1.0:1789760797.611935:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5961 00000100:00100000:1.0:1789760797.611944:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778826882331 time 778888479437 state 0x2 ref 0 00000100:00100000:3.0:1789760797.624752:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.624938:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397248 00000100:00100000:2.0:1789760797.624947:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397248 00000100:00100000:2.0:1789760797.625000:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778826882331 time 778888479437 state 0x0 ref 1 00000100:00100000:2.0:1789760797.625016:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778888739853 time 778888479437 state 0x1 ref 1 00000100:00100000:2.0:1789760797.625039:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778888739853 time 778940343054 state 0x0 ref 1 00000100:00100000:2.0:1789760797.625052:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507397248:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.625172:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24297472, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.625181:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.625197:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760797.625201:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.625207:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.625221:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.689053:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:1.0:1789760797.689073:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689087:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689097:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689105:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.689110:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689119:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689139:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689146:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f490570f/0x9638aab0164b9e6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.689240:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108014700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507397248:12345-192.168.203.41@tcp:4:dd.0 Request processed in 64186us (64547us total) trans 4294973252 rc 0/0 00000100:00100000:1.0:1789760797.689251:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5962 00000100:00100000:1.0:1789760797.689260:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778888739853 time 778940343054 state 0x2 ref 0 00000001:00080000:0.0:1789760797.690401:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00000100:00100000:3.0:1789760797.692339:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.692513:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397376 00000100:00100000:1.0:1789760797.692523:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397376 00000100:00100000:1.0:1789760797.692579:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778888739853 time 778940343054 state 0x0 ref 1 00000100:00100000:1.0:1789760797.692595:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778940603470 time 778940343054 state 0x1 ref 1 00000100:00100000:1.0:1789760797.692617:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778940603470 time 779007920992 state 0x0 ref 1 00000100:00100000:1.0:1789760797.692629:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507397376:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.692677:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24301568, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.692685:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.692811:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.692815:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.692831:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.692846:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.720931:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00002000:00010000:2.0:1789760797.721015:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:2.0:1789760797.721031:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721044:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721054:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721062:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da200 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.721067:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721077:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721096:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721104:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054bd0839/0x9638aab0164b9e76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.721230:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108015500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507397376:12345-192.168.203.41@tcp:4:dd.0 Request processed in 28598us (28895us total) trans 4294973253 rc 0/0 00000100:00100000:2.0:1789760797.721241:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5963 00000100:00100000:2.0:1789760797.721252:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778940603470 time 779007920992 state 0x2 ref 0 00000100:00100000:2.0:1789760797.725320:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.725424:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397504 00000100:00100000:1.0:1789760797.725433:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397504 00000100:00100000:1.0:1789760797.725487:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 778940603470 time 779007920992 state 0x0 ref 1 00000100:00100000:1.0:1789760797.725503:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779008181408 time 779007920992 state 0x1 ref 1 00000100:00100000:1.0:1789760797.725536:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779008181408 time 779040839675 state 0x0 ref 1 00000100:00100000:1.0:1789760797.725549:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507397504:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.725597:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24305664, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.725606:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.725621:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.725626:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.725631:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.725644:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.752260:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00002000:00010000:1.0:1789760797.752489:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:1.0:1789760797.752507:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752519:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752528:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752534:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.752540:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752547:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752566:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.752574:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.752662:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108023b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507397504:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27110us (27348us total) trans 4294973254 rc 0/0 00000100:00100000:1.0:1789760797.752672:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5964 00000100:00100000:1.0:1789760797.752681:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779008181408 time 779040839675 state 0x2 ref 0 00000100:00100000:3.0:1789760797.762205:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.762282:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397632 00000100:00100000:1.0:1789760797.762291:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397632 00000100:00100000:1.0:1789760797.762348:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779008181408 time 779040839675 state 0x0 ref 1 00000100:00100000:1.0:1789760797.762364:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779041100091 time 779040839675 state 0x1 ref 1 00000100:00100000:1.0:1789760797.762414:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779041100091 time 779077709956 state 0x0 ref 1 00000100:00100000:1.0:1789760797.762427:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507397632:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.762482:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24309760, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.762492:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.762510:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.762516:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.762523:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.762538:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.811170:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00002000:00010000:1.0:1789760797.812959:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:1.0:1789760797.812980:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.812994:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.813004:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.813011:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.813016:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.813026:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.813053:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.813062:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.813159:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507397632:12345-192.168.203.41@tcp:4:dd.0 Request processed in 50729us (50960us total) trans 4294973255 rc 0/0 00000100:00100000:1.0:1789760797.813169:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5965 00000100:00100000:1.0:1789760797.813178:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779041100091 time 779077709956 state 0x2 ref 0 00000100:00100000:3.0:1789760797.820062:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760797.820154:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397760 00000100:00100000:2.0:1789760797.820165:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397760 00000100:00100000:2.0:1789760797.820222:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779041100091 time 779077709956 state 0x0 ref 1 00000100:00100000:2.0:1789760797.820237:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779077970372 time 779077709956 state 0x1 ref 1 00000100:00100000:2.0:1789760797.820267:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779077970372 time 779135569831 state 0x0 ref 1 00000100:00100000:2.0:1789760797.820279:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507397760:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760797.820327:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24313856, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:2.0:1789760797.820336:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.820352:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760797.820356:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.820362:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.820386:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.846527:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00002000:00010000:2.0:1789760797.846633:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:2.0:1789760797.846648:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846662:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846671:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846678:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760797.846683:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846723:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846750:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.846759:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9e8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.846980:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107e94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507397760:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26698us (26923us total) trans 4294973256 rc 0/0 00000100:00100000:2.0:1789760797.846992:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5966 00000100:00100000:2.0:1789760797.847001:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779077970372 time 779135569831 state 0x2 ref 0 00000100:00100000:2.0:1789760797.852305:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.852642:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507397888 00000100:00100000:1.0:1789760797.852653:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507397888 00000100:00100000:1.0:1789760797.852728:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779077970372 time 779135569831 state 0x0 ref 1 00000100:00100000:1.0:1789760797.852743:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779135830247 time 779135569831 state 0x1 ref 1 00000100:00100000:1.0:1789760797.852774:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779135830247 time 779168076752 state 0x0 ref 1 00000100:00100000:1.0:1789760797.852789:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507397888:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.852841:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24317952, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.852850:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.852865:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.852871:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.852878:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.852970:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.882798:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24317952 -> 24322048 00010000:00010000:1.0:1789760797.882818:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882831:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882841:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882848:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082da400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.882853:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882863:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882883:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.882896:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db8cb55/0x9638aab0164b9e92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.882995:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507397888:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30205us (30696us total) trans 4294973257 rc 0/0 00000100:00100000:1.0:1789760797.883005:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5967 00000100:00100000:1.0:1789760797.883015:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779135830247 time 779168076752 state 0x2 ref 0 00000001:00080000:3.0:1789760797.883186:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00000100:00100000:3.0:1789760797.886214:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.886622:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398016 00000100:00100000:1.0:1789760797.886633:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398016 00000100:00100000:1.0:1789760797.886707:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779135830247 time 779168076752 state 0x0 ref 1 00000100:00100000:1.0:1789760797.886723:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779168337168 time 779168076752 state 0x1 ref 1 00000100:00100000:1.0:1789760797.886745:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779168337168 time 779202048288 state 0x0 ref 1 00000100:00100000:1.0:1789760797.886758:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398016:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.886940:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24322048, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.886950:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.886966:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.886971:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.886978:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.886999:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.943860:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00002000:00010000:1.0:1789760797.945758:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:1.0:1789760797.945775:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945789:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945799:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945806:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760797.945810:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945820:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945838:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945845:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9e99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.945990:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398016:12345-192.168.203.41@tcp:4:dd.0 Request processed in 59230us (59781us total) trans 4294973258 rc 0/0 00000100:00100000:1.0:1789760797.946000:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5968 00000100:00100000:1.0:1789760797.946009:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779168337168 time 779202048288 state 0x2 ref 0 00000100:00100000:2.0:1789760797.951428:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760797.951659:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398144 00000100:00100000:0.0:1789760797.951669:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398144 00000100:00100000:0.0:1789760797.951930:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779168337168 time 779202048288 state 0x0 ref 1 00000100:00100000:0.0:1789760797.951943:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779202308704 time 779202048288 state 0x1 ref 1 00000100:00100000:0.0:1789760797.951966:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779202308704 time 779267269068 state 0x0 ref 1 00000100:00100000:0.0:1789760797.951979:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507398144:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760797.952029:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24326144, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:0.0:1789760797.952038:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.952068:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760797.952073:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.952079:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.952093:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.981464:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00002000:00010000:0.0:1789760797.981838:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:0.0:1789760797.981853:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981866:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981876:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981883:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760797.981887:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981896:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981916:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.981926:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb7c9219/0x9638aab0164b9ea0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.982016:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108021180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507398144:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30035us (30593us total) trans 4294973259 rc 0/0 00000100:00100000:0.0:1789760797.982027:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5969 00000100:00100000:0.0:1789760797.982037:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779202308704 time 779267269068 state 0x2 ref 0 00000100:00100000:2.0:1789760797.986342:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760797.986473:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398272 00000100:00100000:1.0:1789760797.986572:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398272 00000100:00100000:1.0:1789760797.986629:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779202308704 time 779267269068 state 0x0 ref 1 00000100:00100000:1.0:1789760797.986644:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779267529484 time 779267269068 state 0x1 ref 1 00000100:00100000:1.0:1789760797.986673:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779267529484 time 779301976587 state 0x0 ref 1 00000100:00100000:1.0:1789760797.986685:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398272:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760797.986767:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24330240, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760797.986777:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.986793:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760797.986797:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.986803:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.986816:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.036006:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:1.0:1789760798.036025:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036038:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036047:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036054:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760798.036059:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036068:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036088:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.036096:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9ea7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.036204:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108020a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507398272:12345-192.168.203.41@tcp:4:dd.0 Request processed in 49515us (49927us total) trans 4294973260 rc 0/0 00000100:00100000:1.0:1789760798.036218:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5970 00000100:00100000:1.0:1789760798.036228:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779267529484 time 779301976587 state 0x2 ref 0 00000001:00080000:3.0:1789760798.037139:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00000100:00100000:3.0:1789760798.046008:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.046104:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398400 00000100:00100000:1.0:1789760798.046114:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398400 00000100:00100000:1.0:1789760798.046167:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779267529484 time 779301976587 state 0x0 ref 1 00000100:00100000:1.0:1789760798.046182:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779302237003 time 779301976587 state 0x1 ref 1 00000100:00100000:1.0:1789760798.046213:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779302237003 time 779361516142 state 0x0 ref 1 00000100:00100000:1.0:1789760798.046226:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2416:x1876699507398400:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.046272:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24334336, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc007 00010000:00010000:1.0:1789760798.046281:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.046297:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.046301:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.046308:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.046324:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.109219:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24334336 -> 24338432 00002000:00010000:2.0:1789760798.109229:0:8091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:2.0:1789760798.109232:0:8091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:2.0:1789760798.109245:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109259:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109269:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109278:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.109282:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109292:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109313:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.109321:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080558799/0x9638aab0164b9eae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.109554:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2416:x1876699507398400:12345-192.168.203.41@tcp:4:dd.0 Request processed in 63325us (63551us total) trans 4294973261 rc 0/0 00000100:00100000:2.0:1789760798.109565:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5971 00000100:00100000:2.0:1789760798.109575:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779302237003 time 779361516142 state 0x2 ref 0 00000100:00100000:3.0:1789760798.113322:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.113446:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398528 00000100:00100000:1.0:1789760798.113455:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398528 00000100:00100000:1.0:1789760798.113513:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779302237003 time 779361516142 state 0x0 ref 1 00000100:00100000:1.0:1789760798.113527:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779361776558 time 779361516142 state 0x1 ref 1 00000100:00100000:1.0:1789760798.113558:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779361776558 time 779428860730 state 0x0 ref 1 00000100:00100000:1.0:1789760798.113571:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507398528:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.113619:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24338432, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:1.0:1789760798.113629:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.113645:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.113649:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.113656:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.113670:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.113912:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00000001:00080000:2.0:1789760798.142479:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00002000:00010000:3.0:1789760798.142581:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24338432 -> 24342528 00010000:00010000:3.0:1789760798.142597:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142609:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142624:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142631:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760798.142646:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142656:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142675:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.142683:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.142796:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234d43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398528:12345-192.168.203.41@tcp:4:dd.0 Request processed in 29222us (29479us total) trans 4294973262 rc 0/0 00000100:00100000:3.0:1789760798.142806:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5972 00000100:00100000:3.0:1789760798.142827:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779361776558 time 779428860730 state 0x2 ref 0 00000100:00100000:1.0:1789760798.146867:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760798.146972:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398656 00000100:00100000:2.0:1789760798.146983:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398656 00000100:00100000:2.0:1789760798.147041:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779361776558 time 779428860730 state 0x0 ref 1 00000100:00100000:2.0:1789760798.147056:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779429121146 time 779428860730 state 0x1 ref 1 00000100:00100000:2.0:1789760798.147087:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779429121146 time 779462390081 state 0x0 ref 1 00000100:00100000:2.0:1789760798.147100:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398656:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760798.147149:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24342528, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:2.0:1789760798.147158:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.147173:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760798.147182:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.147188:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.147202:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.189226:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:2.0:1789760798.189245:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189259:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189269:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189277:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.189282:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189293:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189313:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.189321:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002003d6de/0x9638aab0164b9ebc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.189587:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507398656:12345-192.168.203.41@tcp:4:dd.0 Request processed in 42484us (42725us total) trans 4294973263 rc 0/0 00000100:00100000:2.0:1789760798.189599:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5973 00000100:00100000:2.0:1789760798.189609:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779429121146 time 779462390081 state 0x2 ref 0 00000001:00080000:3.0:1789760798.190139:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00000100:00100000:2.0:1789760798.202559:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.202734:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398784 00000100:00100000:1.0:1789760798.202743:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398784 00000100:00100000:1.0:1789760798.202792:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779429121146 time 779462390081 state 0x0 ref 1 00000100:00100000:1.0:1789760798.202804:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779462650497 time 779462390081 state 0x1 ref 1 00000100:00100000:1.0:1789760798.202830:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779462650497 time 779518133330 state 0x0 ref 1 00000100:00100000:1.0:1789760798.202842:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507398784:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.202894:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24346624, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:1.0:1789760798.202904:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.202921:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.202925:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.202932:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.202946:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.241987:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24346624 -> 24350720 00010000:00010000:1.0:1789760798.242007:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242021:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242031:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242037:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760798.242042:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242052:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242073:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.242080:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ec3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.242175:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+7:2417:x1876699507398784:12345-192.168.203.41@tcp:4:dd.0 Request processed in 39329us (39619us total) trans 4294973264 rc 0/0 00000100:00100000:1.0:1789760798.242185:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5974 00000100:00100000:1.0:1789760798.242196:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779462650497 time 779518133330 state 0x2 ref 0 00000001:00080000:3.0:1789760798.242677:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00000100:00100000:1.0:1789760798.251097:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760798.251171:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507398912 00000100:00100000:3.0:1789760798.251181:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507398912 00000100:00100000:3.0:1789760798.251234:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779462650497 time 779518133330 state 0x0 ref 1 00000100:00100000:3.0:1789760798.251250:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779518393746 time 779518133330 state 0x1 ref 1 00000100:00100000:3.0:1789760798.251278:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779518393746 time 779566581924 state 0x0 ref 1 00000100:00100000:3.0:1789760798.251290:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507398912:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760798.251340:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24350720, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:3.0:1789760798.251350:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.251375:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760798.251381:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.251398:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.251420:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.298889:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00002000:00010000:1.0:1789760798.298992:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:1.0:1789760798.299008:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299021:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299030:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299043:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760798.299048:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299057:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299077:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.299084:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9eca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.299184:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507398912:12345-192.168.203.41@tcp:4:dd.0 Request processed in 47889us (48089us total) trans 4294973265 rc 0/0 00000100:00100000:1.0:1789760798.299194:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5975 00000100:00100000:1.0:1789760798.299204:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779518393746 time 779566581924 state 0x2 ref 0 00000100:00100000:2.0:1789760798.303842:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.303968:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399040 00000100:00100000:1.0:1789760798.303978:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399040 00000100:00100000:1.0:1789760798.304034:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779518393746 time 779566581924 state 0x0 ref 1 00000100:00100000:1.0:1789760798.304050:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779566842340 time 779566581924 state 0x1 ref 1 00000100:00100000:1.0:1789760798.304081:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779566842340 time 779619383881 state 0x0 ref 1 00000100:00100000:1.0:1789760798.304097:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399040:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.304146:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24354816, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:1.0:1789760798.304154:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.304170:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.304175:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.304181:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.304196:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.339498:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00002000:00010000:0.0:1789760798.339598:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:0.0:1789760798.339634:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339647:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339656:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339663:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082daa00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760798.339667:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339676:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339720:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.339729:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e0c422/0x9638aab0164b9ed1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.339813:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c108022300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399040:12345-192.168.203.41@tcp:4:dd.0 Request processed in 35713us (35978us total) trans 4294973266 rc 0/0 00000100:00100000:0.0:1789760798.339824:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5976 00000100:00100000:0.0:1789760798.339834:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779566842340 time 779619383881 state 0x2 ref 0 00000100:00100000:0.0:1789760798.345689:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.345819:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399168 00000100:00100000:1.0:1789760798.345829:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399168 00000100:00100000:1.0:1789760798.345877:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779566842340 time 779619383881 state 0x0 ref 1 00000100:00100000:1.0:1789760798.345892:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779619644297 time 779619383881 state 0x1 ref 1 00000100:00100000:1.0:1789760798.346013:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779619644297 time 779661315784 state 0x0 ref 1 00000100:00100000:1.0:1789760798.346026:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507399168:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.346067:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24358912, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:1.0:1789760798.346075:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.346091:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.346095:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.346101:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.346113:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.382168:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00002000:00010000:1.0:1789760798.382255:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:1.0:1789760798.382271:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382284:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382294:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382302:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:1.0:1789760798.382307:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382317:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382338:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.382347:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9ed8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.382542:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507399168:12345-192.168.203.41@tcp:4:dd.0 Request processed in 36511us (36855us total) trans 4294973267 rc 0/0 00000100:00100000:1.0:1789760798.382553:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5977 00000100:00100000:1.0:1789760798.382563:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779619644297 time 779661315784 state 0x2 ref 0 00000100:00100000:1.0:1789760798.392172:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760798.392266:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399296 00000100:00100000:2.0:1789760798.392276:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399296 00000100:00100000:2.0:1789760798.392332:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779619644297 time 779661315784 state 0x0 ref 1 00000100:00100000:2.0:1789760798.392349:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779661576200 time 779661315784 state 0x1 ref 1 00000100:00100000:2.0:1789760798.392386:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779661576200 time 779707688956 state 0x0 ref 1 00000100:00100000:2.0:1789760798.392630:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507399296:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760798.392684:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24363008, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:2.0:1789760798.392767:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.392783:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760798.392787:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.392793:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.392806:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.418553:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00002000:00010000:2.0:1789760798.418665:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:2.0:1789760798.418680:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418709:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418719:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418726:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5400 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.418731:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418740:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418760:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.418768:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088ebe1ef/0x9638aab0164b9edf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.418892:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507399296:12345-192.168.203.41@tcp:4:dd.0 Request processed in 26491us (26726us total) trans 4294973268 rc 0/0 00000100:00100000:2.0:1789760798.418902:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5978 00000100:00100000:2.0:1789760798.418911:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779661576200 time 779707688956 state 0x2 ref 0 00000100:00100000:2.0:1789760798.422454:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.422539:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399424 00000100:00100000:0.0:1789760798.422549:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399424 00000100:00100000:0.0:1789760798.422604:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779661576200 time 779707688956 state 0x0 ref 1 00000100:00100000:0.0:1789760798.422620:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779707949372 time 779707688956 state 0x1 ref 1 00000100:00100000:0.0:1789760798.422649:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779707949372 time 779737952006 state 0x0 ref 1 00000100:00100000:0.0:1789760798.422661:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507399424:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.422743:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24367104, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.422752:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.422767:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.422771:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.422777:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.422791:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760798.453280:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:0.0:1789760798.453299:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453311:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453320:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453327:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760798.453331:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453340:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453359:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.453379:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ee6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.453475:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2418:x1876699507399424:12345-192.168.203.41@tcp:4:dd.0 Request processed in 30812us (31025us total) trans 4294973269 rc 0/0 00000100:00100000:0.0:1789760798.453485:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5979 00000100:00100000:0.0:1789760798.453494:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779707949372 time 779737952006 state 0x2 ref 0 00000001:00080000:0.0:1789760798.453533:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00000100:00100000:0.0:1789760798.456879:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:1.0:1789760798.456997:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399552 00000100:00100000:1.0:1789760798.457006:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399552 00000100:00100000:1.0:1789760798.457059:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779707949372 time 779737952006 state 0x0 ref 1 00000100:00100000:1.0:1789760798.457074:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779738212422 time 779737952006 state 0x1 ref 1 00000100:00100000:1.0:1789760798.457107:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779738212422 time 779772410248 state 0x0 ref 1 00000100:00100000:1.0:1789760798.457120:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399552:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:1.0:1789760798.457166:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24371200, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:1.0:1789760798.457175:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.457192:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760798.457196:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.457203:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.457218:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.490007:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:3.0:1789760798.490100:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24371200 -> 24375296 00010000:00010000:3.0:1789760798.490116:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490126:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490135:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490143:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c1082db600 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760798.490148:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490157:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490176:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.490185:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5bf386/0x9638aab0164b9eed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.490280:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c234bf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399552:12345-192.168.203.41@tcp:4:dd.0 Request processed in 33159us (33407us total) trans 4294973270 rc 0/0 00000100:00100000:3.0:1789760798.490291:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5980 00000100:00100000:3.0:1789760798.490301:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779738212422 time 779772410248 state 0x2 ref 0 00000100:00100000:3.0:1789760798.493304:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.493417:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399680 00000100:00100000:0.0:1789760798.493427:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399680 00000100:00100000:0.0:1789760798.493479:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779738212422 time 779772410248 state 0x0 ref 1 00000100:00100000:0.0:1789760798.493494:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779772670664 time 779772410248 state 0x1 ref 1 00000100:00100000:0.0:1789760798.493525:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779772670664 time 779808828054 state 0x0 ref 1 00000100:00100000:0.0:1789760798.493538:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507399680:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.493584:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24375296, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.493593:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.493608:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.493613:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.493618:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.493629:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.517051:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00002000:00010000:0.0:1789760798.517160:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24375296 -> 24379392 00010000:00010000:0.0:1789760798.517176:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517189:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517197:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517204:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760798.517208:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517217:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517236:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.517248:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9ef4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.517382:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507399680:12345-192.168.203.41@tcp:4:dd.0 Request processed in 23841us (24083us total) trans 4294973271 rc 0/0 00000100:00100000:0.0:1789760798.517392:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5981 00000100:00100000:0.0:1789760798.517409:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779772670664 time 779808828054 state 0x2 ref 0 00000100:00100000:1.0:1789760798.521044:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.521129:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399808 00000100:00100000:0.0:1789760798.521138:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399808 00000100:00100000:0.0:1789760798.521190:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779772670664 time 779808828054 state 0x0 ref 1 00000100:00100000:0.0:1789760798.521205:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779809088470 time 779808828054 state 0x1 ref 1 00000100:00100000:0.0:1789760798.521275:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779809088470 time 779836577775 state 0x0 ref 1 00000100:00100000:0.0:1789760798.521289:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399808:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.521338:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24379392, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.521347:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.521370:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.521375:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.521381:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.521407:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760798.548714:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:0.0:1789760798.548735:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548747:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548755:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548771:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:0.0:1789760798.548776:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548785:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548805:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.548813:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9efb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.548911:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+7:2419:x1876699507399808:12345-192.168.203.41@tcp:4:dd.0 Request processed in 27619us (27870us total) trans 4294973272 rc 0/0 00000100:00100000:0.0:1789760798.548921:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5982 00000100:00100000:0.0:1789760798.548930:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779809088470 time 779836577775 state 0x2 ref 0 00000001:00080000:0.0:1789760798.548967:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00000100:00100000:2.0:1789760798.554976:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760798.555244:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507399936 00000100:00100000:3.0:1789760798.555255:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507399936 00000100:00100000:3.0:1789760798.555320:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779809088470 time 779836577775 state 0x0 ref 1 00000100:00100000:3.0:1789760798.555336:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779836838191 time 779836577775 state 0x1 ref 1 00000100:00100000:3.0:1789760798.555374:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779836838191 time 779870677246 state 0x0 ref 1 00000100:00100000:3.0:1789760798.555387:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399936:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760798.555448:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24383488, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:3.0:1789760798.555457:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.555472:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760798.555476:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.555482:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.555495:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.589661:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00002000:00010000:3.0:1789760798.589783:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:3.0:1789760798.589798:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589854:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589864:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589872:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c2013a3e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760798.589876:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589886:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589907:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.589914:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c100a78e/0x9638aab0164b9f02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.590205:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507399936:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34615us (35035us total) trans 4294973273 rc 0/0 00000100:00100000:3.0:1789760798.590216:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5983 00000100:00100000:3.0:1789760798.590226:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779836838191 time 779870677246 state 0x2 ref 0 00000100:00100000:3.0:1789760798.596991:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.597094:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400064 00000100:00100000:0.0:1789760798.597103:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400064 00000100:00100000:0.0:1789760798.597152:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779836838191 time 779870677246 state 0x0 ref 1 00000100:00100000:0.0:1789760798.597168:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779870937662 time 779870677246 state 0x1 ref 1 00000100:00100000:0.0:1789760798.597190:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779870937662 time 779912493788 state 0x0 ref 1 00000100:00100000:0.0:1789760798.597203:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507400064:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.597250:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24387584, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.597259:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.597273:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.597278:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.597284:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.597302:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.621528:0:6428:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760798.631122:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00002000:00010000:2.0:1789760798.631229:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:2.0:1789760798.631244:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631257:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631266:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631272:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.631277:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631286:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631303:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.631311:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef99ec0c/0x9638aab0164b9f09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.631423:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c107c78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507400064:12345-192.168.203.41@tcp:4:dd.0 Request processed in 34212us (34432us total) trans 4294973274 rc 0/0 00000100:00100000:2.0:1789760798.631433:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5984 00000100:00100000:2.0:1789760798.631442:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779870937662 time 779912493788 state 0x2 ref 0 00000100:00100000:1.0:1789760798.635024:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.635287:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400192 00000100:00100000:0.0:1789760798.635297:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400192 00000100:00100000:0.0:1789760798.635351:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779870937662 time 779912493788 state 0x0 ref 1 00000100:00100000:0.0:1789760798.635375:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779912754204 time 779912493788 state 0x1 ref 1 00000100:00100000:0.0:1789760798.635405:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779912754204 time 779950708162 state 0x0 ref 1 00000100:00100000:0.0:1789760798.635418:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507400192:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.635466:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24391680, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.635475:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.635490:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.635495:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.635501:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.635580:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.677006:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:2.0:1789760798.677027:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.677035:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00010000:00010000:2.0:1789760798.677040:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.677048:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.677061:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0c00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.677067:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.677134:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.677152:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.677159:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9c8446d/0x9638aab0164b9f10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.677536:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507400192:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41834us (42233us total) trans 4294973275 rc 0/0 00000100:00100000:2.0:1789760798.677546:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5985 00000100:00100000:2.0:1789760798.677556:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779912754204 time 779950708162 state 0x2 ref 0 00000100:00100000:1.0:1789760798.681755:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:0.0:1789760798.683415:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400320 00000100:00100000:0.0:1789760798.683431:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400320 00000100:00100000:0.0:1789760798.683498:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779912754204 time 779950708162 state 0x0 ref 1 00000100:00100000:0.0:1789760798.683513:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779950968578 time 779950708162 state 0x1 ref 1 00000100:00100000:0.0:1789760798.683546:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779950968578 time 779998849513 state 0x0 ref 1 00000100:00100000:0.0:1789760798.683558:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507400320:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.683608:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24395776, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.683617:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.683634:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760798.683638:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.683644:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.683657:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.725081:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:3.0:1789760798.725237:0:8091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:3.0:1789760798.725254:0:8091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725267:0:8091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725276:0:8091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725288:0:8091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc1800 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:3.0:1789760798.725292:0:8091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725300:0:8091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725320:0:8091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.725327:0:8091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc4abda2/0x9638aab0164b9f17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.725521:0:8091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c1083eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2417:x1876699507400320:12345-192.168.203.41@tcp:4:dd.0 Request processed in 41960us (43771us total) trans 4294973276 rc 0/0 00000100:00100000:3.0:1789760798.725531:0:8091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5986 00000100:00100000:3.0:1789760798.725541:0:8091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779950968578 time 779998849513 state 0x2 ref 0 00000100:00100000:3.0:1789760798.729804:0:3456:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760798.730018:0:14618:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400448 00000100:00100000:2.0:1789760798.730028:0:14618:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400448 00000100:00100000:2.0:1789760798.730084:0:14618:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779950968578 time 779998849513 state 0x0 ref 1 00000100:00100000:2.0:1789760798.730098:0:14618:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779999109929 time 779998849513 state 0x1 ref 1 00000100:00100000:2.0:1789760798.730128:0:14618:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779999109929 time 780045431459 state 0x0 ref 1 00000100:00100000:2.0:1789760798.730139:0:14618:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507400448:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:2.0:1789760798.730186:0:14618:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24399872, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:2.0:1789760798.730194:0:14618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.730210:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760798.730214:0:14618:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.730220:0:14618:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.730233:0:14618:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.845236:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00002000:00010000:2.0:1789760798.845437:0:14618:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:2.0:1789760798.845452:0:14618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845464:0:14618:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845471:0:14618:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845478:0:14618:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c107fa5e00 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.845482:0:14618:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845491:0:14618:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845512:0:14618:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.845520:0:14618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e473d0/0x9638aab0164b9f1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.845621:0:14618:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507400448:12345-192.168.203.41@tcp:4:dd.0 Request processed in 115478us (115821us total) trans 4294973277 rc 0/0 00000100:00100000:2.0:1789760798.845631:0:14618:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5987 00000100:00100000:2.0:1789760798.845652:0:14618:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779999109929 time 780045431459 state 0x2 ref 0 00000100:00100000:1.0:1789760798.861893:0:3454:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:2.0:1789760798.863577:0:14616:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400576 00000100:00100000:2.0:1789760798.863590:0:14616:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400576 00000100:00100000:0.0:1789760798.863910:0:14616:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 779999109929 time 780045431459 state 0x0 ref 1 00000100:00100000:0.0:1789760798.863925:0:14616:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780045691875 time 780045431459 state 0x1 ref 1 00000100:00100000:0.0:1789760798.863994:0:14616:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780045691875 time 780179296943 state 0x0 ref 1 00000100:00100000:0.0:1789760798.864012:0:14616:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507400576:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:0.0:1789760798.864061:0:14616:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24403968, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:0.0:1789760798.864070:0:14616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.864085:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760798.864089:0:14616:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.864095:0:14616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.864109:0:14616:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.926198:0:8113:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:2.0:1789760798.926376:0:14616:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:2.0:1789760798.926392:0:14616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926413:0:14616:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926423:0:14616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926436:0:14616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff88c22ddc0000 already unused, calling callback (ffffffffc15dc350) 00010000:00010000:2.0:1789760798.926441:0:14616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926451:0:14616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926471:0:14616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.926478:0:14616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005ce098a/0x9638aab0164b9f25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.926567:0:14616:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff88c240fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:7fdb621d-3376-4597-a0ab-d89800273221+5:2418:x1876699507400576:12345-192.168.203.41@tcp:4:dd.0 Request processed in 62554us (64680us total) trans 4294973278 rc 0/0 00000100:00100000:2.0:1789760798.926578:0:14616:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.41@tcp, seq: 5988 00000100:00100000:2.0:1789760798.926587:0:14616:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780045691875 time 780179296943 state 0x2 ref 0 00000100:00100000:1.0:1789760798.932065:0:3455:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.41@tcp (source: 12345-192.168.203.41@tcp) 00000100:00100000:3.0:1789760798.932226:0:8091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699507400704 00000100:00100000:3.0:1789760798.932237:0:8091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699507400704 00000100:00100000:3.0:1789760798.932292:0:8091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780045691875 time 780179296943 state 0x0 ref 1 00000100:00100000:3.0:1789760798.932307:0:8091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780179557359 time 780179296943 state 0x1 ref 1 00000100:00100000:3.0:1789760798.932336:0:8091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff88c234e84000 key nid={192.168.203.41@tcp}&opcode={ost_write} rule@0xffff88c10432a400 gen 0 rate 15 depth 3 token 3 deadline 780179557359 time 780247639371 state 0x0 ref 1 00000100:00100000:3.0:1789760798.932350:0:8091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff88c240fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:7fdb621d-3376-4597-a0ab-d89800273221+5:2419:x1876699507400704:12345-192.168.203.41@tcp:4:dd.0 00002000:00010000:3.0:1789760798.932417:0:8091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24408064, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc007 00010000:00010000:3.0:1789760798.932427:0:8091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9f2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.932443:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760798.932447:0:8091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.932453:0:8091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9f2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.932468:0:8091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab4f5ca3/0x9638aab0164b9f2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 57410 lines, 57410 kept, 0 dropped, 0 bad.